-- Logs begin at Thu 2022-09-22 14:16:41 UTC, end at Thu 2022-09-22 15:29:15 UTC. -- Sep 22 14:43:21.979035 dv-cinder-856463-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 22 14:43:21.986498 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 22 14:43:21 2022] *** Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: nodename: dv-cinder-856463-5-iscsi Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: machine: x86_64 Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: clock source: unix Sep 22 14:43:21.988546 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: pcre jit disabled Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: detected number of CPU cores: 4 Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: current working directory: / Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: detected binary path: /usr/bin/uwsgi-core Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: your processes number limit is 47680 Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: your memory page size is 4096 bytes Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: detected max file descriptor number: 2048 Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: lock engine: pthread robust mutexes Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: thunder lock: enabled Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 22 14:43:21.989013 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 22 14:43:22.029317 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: Python main interpreter initialized at 0x5623f5e7c9c0 Sep 22 14:43:22.029317 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: python threads support enabled Sep 22 14:43:22.029317 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: your server socket listen backlog is limited to 100 connections Sep 22 14:43:22.029317 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: your mercy for graceful operations on workers is 90 seconds Sep 22 14:43:22.029708 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: mapped 403077 bytes (393 KB) for 2 cores Sep 22 14:43:22.029791 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: *** Operational MODE: preforking *** Sep 22 14:43:22.030046 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: *** uWSGI is running in multiple interpreter mode *** Sep 22 14:43:22.030046 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: spawned uWSGI master process (pid: 81440) Sep 22 14:43:22.030249 dv-cinder-856463-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 22 14:43:22.031377 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: spawned uWSGI worker 1 (pid: 81451, cores: 1) Sep 22 14:43:22.031377 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: spawned uWSGI worker 2 (pid: 81452, cores: 1) Sep 22 14:43:22.031377 dv-cinder-856463-5-iscsi devstack@keystone.service[81440]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 22 14:43:26.297084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:26.297084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: from cryptography.utils import int_from_bytes Sep 22 14:43:26.297876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:26.297876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: from cryptography.utils import int_from_bytes Sep 22 14:43:26.298142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:26.298142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: from cryptography.utils import int_from_bytes Sep 22 14:43:26.298594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:26.298594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: from cryptography.utils import int_from_bytes Sep 22 14:43:26.347475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 14:43:26.347645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 14:43:26.347840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] command line args: [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 14:43:26.348104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 14:43:26.348278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ================================================================================ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 14:43:26.348452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 14:43:26.348537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] admin_token = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.348648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 14:43:26.348758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config_dir = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.348841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] command line args: [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 14:43:26.348944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 14:43:26.349187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config_source = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ================================================================================ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 14:43:26.349363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] debug = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] admin_token = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config_dir = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.349991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config_source = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] debug = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.350914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_config_append = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_dir = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_config_append = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.351988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_dir = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.352963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.353987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.354063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.354177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.354258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.354442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.354712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] publish_errors = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.355883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] standard_threads = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.356921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] publish_errors = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_journal = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_json = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_stderr = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.357978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_syslog = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.358061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.358207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.358284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.358461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.358542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] standard_threads = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.358690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.358772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.358936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.359016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.359115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.359217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.359323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_journal = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.359428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.359504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_json = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.359662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.359750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_stderr = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.359822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_syslog = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.359903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:26.360131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.360888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.361039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.361123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.361283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.361857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.362833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.363988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.364940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.365951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.366894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.367856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.368997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.369898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.370954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.371903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.372955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.373971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.374995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.375979 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.376967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.377697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.377787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.377941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.378963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.379930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.380870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.381946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.382922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.383854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.384974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.external = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.385920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.password = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.token = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.386933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.external = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.387908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.password = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.token = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.388855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.389954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.390922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.391984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.392922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.393949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.394109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.394195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.394410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.394669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.394865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.395083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.395083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.395568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.395778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.395992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.396199 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.396412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.396615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.396820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.397024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.397224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.397415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.397713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.397918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.398121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.398332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.398536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.398753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.398910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.399991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.400049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.400107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.400165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.400230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.400619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.400836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.401047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.401246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.401452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.401657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.401823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.401955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.402874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.403973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.404960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.405919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.406469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.406686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.406886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.407092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.407323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.407528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.407528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.407936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.408985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.409940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.410947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.411178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.411314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.411314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.411550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.411715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.411715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.412118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.412326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.412326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.412709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.412912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.412912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.413264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.413447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.413643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.413833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.414893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.415909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.416904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.417959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.418973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.419987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.420883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.421910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.422982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.423781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.424987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.425928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.426940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427362 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427512 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.427951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.428992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.429973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.430886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.431966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.432977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433362 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.433885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.434914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 14:43:26.435681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.435868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.436018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.436255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.436492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.436728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.436973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.437209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.437453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.437690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.437938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.438176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.438419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.438651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.438893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.439130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:26.439325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 14:43:26.464387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.464606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.464825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.467852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.468085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.468313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.473274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.476703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.501360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.501583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.503697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.503938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.555117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.555726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.603053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.603301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.604370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.604614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.762192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.762428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.762638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.762723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.762856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.762941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.763070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.763153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.763308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.763407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.763527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.763746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.786468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.786714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.786936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.787849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.788069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.788069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.788295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.798710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.799665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.812869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.813004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.813088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.813211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.813293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.813422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.813635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:26.829699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.829906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:26.830233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.830316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.830464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:26.830602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:26.830719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.830801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.830975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:26.831059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.831250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.831334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.831510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.831590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.831768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.832028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.840996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.841269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.841536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.841619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.841816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.841913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.842087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.842371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.849304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.849938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.852201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.852483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.852759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.852865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.853034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.853152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.853307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.853437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.853559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.853724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.853825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.854899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.855090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.855391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.855664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.877782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.878764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.879260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.879548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.880315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.880606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.884424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.884703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.884978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.885526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.885814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.886109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.891256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.891541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.892457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.892777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.897308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.897607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.897894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.898182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.898456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.898610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.898731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.898917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.898999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.899263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.899428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.899575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.899905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.900194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.900495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.900802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.915413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.915687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.916966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.917256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.919521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.921118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.922766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.923097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.924662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.924981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.927427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.929610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.930794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.932932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.934109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.934402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.934685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.934965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.935257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.935552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.936211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.936512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.936802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.937091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.937374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.937676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.951730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.952014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.952279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.952539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.953990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.954278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.954551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.954819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.959352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.960869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.961784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.962374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.962647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.963352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.964890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.965163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.965405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.965692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.965970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.966249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.966631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.967993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.968283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.968566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.968858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.969250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.978165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.978447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.978725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.979005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.979408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.979696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.979972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.980910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.981194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.981476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.981757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.982137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.982420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.982719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:26.995168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.998076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:26.998220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.000956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.001113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.002470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.003944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.004094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.004233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.004605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.005417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.006989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.007301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.007708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.011106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.012561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.014294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.015431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.015765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.015896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.016087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.016362 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.019162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.019577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.019866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.020152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.024599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.024890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.025170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.027933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.028229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.028509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.031977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.032258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.032533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.032803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.033071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.033429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.033701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.033974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.034238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.034509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.034775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.035048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.035333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.035430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:27.035614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.035697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.035971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.036248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.036522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.036795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.037073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.037352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.037623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.037910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.038183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.038462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.038732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.039000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:27.070682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5623f5e7c9c0 pid: 81452 (default app) Sep 22 14:43:27.074075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac91f4-b57d-4a06-b5ce-b7984e37a79b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:27.074358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac91f4-b57d-4a06-b5ce-b7984e37a79b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:27.074628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac91f4-b57d-4a06-b5ce-b7984e37a79b None None] PATH_INFO: `/v3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:27.074794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5623f5e7c9c0 pid: 81451 (default app) Sep 22 14:43:27.078750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Thu Sep 22 14:43:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:34.062924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:34.063501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:34.063501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:34.067718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1/2] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:34.073293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:34.073803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:34.073898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:34.152562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: INFO dbcounter [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] Registered counter for database keystone Sep 22 14:43:34.175848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG oslo_db.sqlalchemy.engines [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81452) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 14:43:34.182886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writer thread running {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 14:43:34.365401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG passlib.handlers.bcrypt [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81452) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 14:43:34.365678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG passlib.handlers.bcrypt [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81452) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 14:43:34.424171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:34.425759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:34.565466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 2/3] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 22 14:43:35.026164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:35.097384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: INFO dbcounter [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] Registered counter for database keystone Sep 22 14:43:35.119854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG oslo_db.sqlalchemy.engines [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81451) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 14:43:35.127485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writer thread running {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 14:43:35.247226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6bf5b-ee12-42c1-934f-d702b7149b9c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:35.251491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:35.252159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:35.252438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:35.252707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:35.261652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:35.314726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 14:43:35.315074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f5c0e-2f84-4a4e-8fe0-d4c444ad1931 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:35.325413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 2/4] 10.222.0.66 () {58 vars in 1324 bytes} [Thu Sep 22 14:43:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 14:43:35.331978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:35.349529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fc94e-0dfc-49f2-ac4c-1c0ae0a09795 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:35.353944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:35.355072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:35.355423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:35.355665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] PATH_INFO: `/v3/projects/0562937a351d4a8397127ac952e02625` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:35.357281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] RBAC: Authorizing `identity:get_project(project_id=0562937a351d4a8397127ac952e02625)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:35.411234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 14:43:35.411580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff720588-4513-4f32-b193-ce2f50f13e23 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:35.416255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 3/5] 10.222.0.66 () {56 vars in 1207 bytes} [Thu Sep 22 14:43:35 2022] GET /identity/v3/projects/0562937a351d4a8397127ac952e02625 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:37.593504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb33b-e8f5-45f5-876f-eddc08362547 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:37.594016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb33b-e8f5-45f5-876f-eddc08362547 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:37.594128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fb33b-e8f5-45f5-876f-eddc08362547 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:37.598264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 3/6] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:37.604079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:37.604537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:37.604621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:37.625084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:37.627235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:37.635443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 4/7] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 22 14:43:37.908552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb33b-e8f5-45f5-876f-eddc08362547 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:37.920860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fb33b-e8f5-45f5-876f-eddc08362547 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:37.926423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae8ffe-2579-4ca0-863e-3496385e9c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:37.927040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8ffe-2579-4ca0-863e-3496385e9c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:37.927359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8ffe-2579-4ca0-863e-3496385e9c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:37.927641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8ffe-2579-4ca0-863e-3496385e9c7a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:37.935987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae8ffe-2579-4ca0-863e-3496385e9c7a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:37.937770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae8ffe-2579-4ca0-863e-3496385e9c7a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:37.946945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 4/8] 10.222.0.66 () {58 vars in 1324 bytes} [Thu Sep 22 14:43:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 14:43:37.952706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:37.962180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0243c4-fec1-4d61-aab3-b641b3c15bf6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:37.966616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5924e-52c1-442f-a0dc-6d89b6c99b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:37.967281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5924e-52c1-442f-a0dc-6d89b6c99b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:37.967564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5924e-52c1-442f-a0dc-6d89b6c99b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:37.967836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5924e-52c1-442f-a0dc-6d89b6c99b65 admin admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:37.969462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5924e-52c1-442f-a0dc-6d89b6c99b65 admin admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:37.971828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5924e-52c1-442f-a0dc-6d89b6c99b65 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:37.973388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 5/9] 10.222.0.66 () {56 vars in 1198 bytes} [Thu Sep 22 14:43:37 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:40.242847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359345-6317-45c1-afd5-9d481e71b077 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.243437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359345-6317-45c1-afd5-9d481e71b077 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.243437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359345-6317-45c1-afd5-9d481e71b077 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.244783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 5/10] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:40.251110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.251634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.251806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.276135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:40.277503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:40.310662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014378f1-ed22-4f5e-a664-dd31a9fe8cc0 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.310947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014378f1-ed22-4f5e-a664-dd31a9fe8cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.311258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014378f1-ed22-4f5e-a664-dd31a9fe8cc0 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.312608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 6/11] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:40.318130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.318424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.318720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.363713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG passlib.handlers.bcrypt [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81451) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 14:43:40.364102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG passlib.handlers.bcrypt [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81451) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 14:43:40.371916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 6/12] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:40.413886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:40.415541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:40.522080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 7/13] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:40.660256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:40.687196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:40.779726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cbbd2-6f79-489a-adc7-c3771c7e5ea1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:40.790025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:40.817050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:40.817704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.818559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.818842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.830985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:40.839970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:40.841176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:43:40.844146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 7/14] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:40.848247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:40.918094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a304a-d2a7-481a-bc81-638f289cbe19 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:40.922317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:40.944094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:40.963857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c677ec-36ee-4165-aa36-b0eab1c9b494 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:40.964313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:40.965195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.965483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.965753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.004278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:41.004896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:41.005174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:41.005447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.006161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:41.006463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.007943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.021351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 8/15] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:41.027081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:41.040444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.042038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.136967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18000c-e1f4-4580-8301-c3410ffe3ac2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:41.156690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 8/16] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:43:41.162418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:41.165330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:41.175344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:41.176051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:41.176338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:41.176637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.177842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.179339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.181084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 9/17] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:41 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:41.186653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:41.206830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:41.276633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287d275-5f0a-4bbc-9ae2-5716e5377400 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:41.299548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ce4d-6726-42d6-bbbe-e86342e1d930 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:41.321036 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:41.321919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:41.322197 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:41.322570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.341501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:41.342147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:41.342428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:41.342698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.342856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.343935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.344247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.345435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.347029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 10/18] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:41.353408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:43:41.364659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:41.365617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:43:41.366394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:43:41.366394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:41.366394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:43:41.366394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 9/19] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:43:41.373262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:41.437011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:41.441668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:41.468509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc83d2-6954-44e6-94fe-cb42e2b94a57 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:41.497178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f717ad8-55fc-4332-b20d-57124ba78205 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:41.518947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:41.519647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:41.519928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:41.520206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.521042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.domain.id=default, user.id=f95719d40ecc461fa97c90ba29b8ba24) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:41.521411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.522987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650f97f-dcb6-4e9f-8e03-95a439a9fce9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.532529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 11/20] 10.222.0.66 () {56 vars in 1322 bytes} [Thu Sep 22 14:43:41 2022] GET /identity/v3/role_assignments?user.id=f95719d40ecc461fa97c90ba29b8ba24&scope.domain.id=default&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 233 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:41.535266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:41.536079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:41.536358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:41.536633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:41.537377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:41.537615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:41.539063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfee4b-fd8f-48ad-bcb5-a89b685f1610 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:41.565056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 10/21] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:44.552743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289679e8-6fc2-41da-9258-1a6cbd7fe2d3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:44.553496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289679e8-6fc2-41da-9258-1a6cbd7fe2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:44.553496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289679e8-6fc2-41da-9258-1a6cbd7fe2d3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:44.555071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 12/22] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:44.560653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33933a2b-5452-4941-90cb-a2a39eceb49a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:44.561116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33933a2b-5452-4941-90cb-a2a39eceb49a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:44.561206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33933a2b-5452-4941-90cb-a2a39eceb49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:44.588621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-33933a2b-5452-4941-90cb-a2a39eceb49a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:44.589956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-33933a2b-5452-4941-90cb-a2a39eceb49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:44.686808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 11/23] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:44.762607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b7b1a-db20-4764-a51f-598337a8a4f5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:44.762880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b7b1a-db20-4764-a51f-598337a8a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:44.763082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b7b1a-db20-4764-a51f-598337a8a4f5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:44.764463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 13/24] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:44.770950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf4f6-eefe-4dd9-b8f4-911935ddc570 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:44.771326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf4f6-eefe-4dd9-b8f4-911935ddc570 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:44.771684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaf4f6-eefe-4dd9-b8f4-911935ddc570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:44.801642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fccaf4f6-eefe-4dd9-b8f4-911935ddc570 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:44.803544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fccaf4f6-eefe-4dd9-b8f4-911935ddc570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:44.841232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5a06f-99f7-4649-96ca-a90410af721d None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:44.841585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5a06f-99f7-4649-96ca-a90410af721d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:44.841789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5a06f-99f7-4649-96ca-a90410af721d None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:44.843189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 14/25] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:44.849064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:44.849380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:44.849618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:44.895558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:44.895947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:44.897184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:44.957666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 12/26] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:45.017870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439566e-db97-4ada-a889-d776f1b59c55 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.018223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439566e-db97-4ada-a889-d776f1b59c55 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.018679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439566e-db97-4ada-a889-d776f1b59c55 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.020889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 13/27] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:45.027767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.028130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.028322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.065898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 15/28] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:45.066886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:45.068294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:45.147077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 14/29] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:45.205009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:45.292382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:45.320710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e20669-1ed3-4937-8ff6-ba98bee8e0c8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:45.370374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:45.371411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.371833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.372249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.390719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:45.400591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:45.401508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:45.402264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:43:45.402816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 16/30] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:45.412888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:45.540133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:45.571476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10795077-6f30-4e03-84a3-3225a1e93973 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:45.591608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:45.629587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:45.630231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.630530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.630834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.631600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:45.631945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:45.633417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:45.664531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 15/31] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:45.666260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:45.726858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1b83a-25d5-4563-81a3-864769708f60 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:45.771980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:45.772980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.773648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.774148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.776733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:45.790178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:45.792155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:45.803191 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eef5aa-a1e0-4cf5-ad75-a2a783ae7853 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:43:45.810912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:45.811894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:43:45.813005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:43:45.813005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:45.813005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:43:45.813005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 17/32] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:43:45.815902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:45.847476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:45.849011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.849507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.849975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.850853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:45.852511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:45.949522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42eca5a-8404-4952-9359-a6564c2d4229 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:45.991752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:45.992427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:45.992749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:45.993086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:45.994254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:45.995811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:45.997412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 18/33] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:45 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:45.998164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.013101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 16/34] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Sep 22 14:43:45 2022] POST /identity/v3/roles => generated 219 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:46.014669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.064666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.077986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.150199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47a858-cc7f-48e7-b35c-d205ac2affdf None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:46.174321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55f975-9865-40e9-80f0-eb05aed75516 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:46.195336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4937713-6fd7-46da-9cf9-49b912a85ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:46.196026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4937713-6fd7-46da-9cf9-49b912a85ef7 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.196316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4937713-6fd7-46da-9cf9-49b912a85ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.196593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4937713-6fd7-46da-9cf9-49b912a85ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.197285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4937713-6fd7-46da-9cf9-49b912a85ef7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:46.199754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4937713-6fd7-46da-9cf9-49b912a85ef7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:46.215044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:46.216547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.216547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.216547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.217154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:46.217470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:46.219311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:46.233827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 17/35] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:46.234700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.332293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 19/36] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Sep 22 14:43:45 2022] POST /identity/v3/roles => generated 225 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:46.333880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cafbe-4750-4535-b208-609600aa1c08 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.334180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cafbe-4750-4535-b208-609600aa1c08 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.334459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cafbe-4750-4535-b208-609600aa1c08 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.338235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 20/37] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:46.338235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff339-2763-441e-87bb-ad6182b4c7f1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.338235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff339-2763-441e-87bb-ad6182b4c7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.338235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff339-2763-441e-87bb-ad6182b4c7f1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.343039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 21/38] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:46.343039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff339-2763-441e-87bb-ad6182b4c7f1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.349839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.370700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d65ff-96f6-4412-b60c-661d69fed73b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:46.411510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82f8eb-c752-422b-be2b-9008c7079a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:46.412083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82f8eb-c752-422b-be2b-9008c7079a51 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.412368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82f8eb-c752-422b-be2b-9008c7079a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.412643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82f8eb-c752-422b-be2b-9008c7079a51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.414256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82f8eb-c752-422b-be2b-9008c7079a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:46.415799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82f8eb-c752-422b-be2b-9008c7079a51 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:46.417501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 18/39] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:46.418847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3010677-e491-4a54-b9e1-6b485ef6a464 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.419130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3010677-e491-4a54-b9e1-6b485ef6a464 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.419429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3010677-e491-4a54-b9e1-6b485ef6a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.442379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f3010677-e491-4a54-b9e1-6b485ef6a464 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:46.444079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3010677-e491-4a54-b9e1-6b485ef6a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:46.472083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff339-2763-441e-87bb-ad6182b4c7f1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:46.492294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.521903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:46.522416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.523561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.523561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.523793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:46.525443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:46.532179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 19/40] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:46.534922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.535034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.535286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.556160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:46.557410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:46.566041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 22/41] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/projects => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:46.566992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.655778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 20/42] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:46.712002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87866e-fdaa-420e-bfeb-337ad6cd8d54 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:46.754954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:46.755745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.755945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.756224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] PATH_INFO: `/v3/domains/default/users/f95719d40ecc461fa97c90ba29b8ba24/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.758255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f95719d40ecc461fa97c90ba29b8ba24, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:46.760103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:46.784420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 23/43] 10.222.0.66 () {58 vars in 1361 bytes} [Thu Sep 22 14:43:46 2022] PUT /identity/v3/domains/default/users/f95719d40ecc461fa97c90ba29b8ba24/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:43:46.851104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.876684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.966608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.979301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b0ae3-4f88-4f4e-8b9e-164e28bb0d25 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.002334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.018932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.019583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.019861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.020132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.020814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.022224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.042396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 21/44] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/roles => generated 223 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:47.202474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e00496-ef78-43fa-9c4e-cfa2832b77fa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.244009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43df12-a167-4c59-a7f5-1fb7ace3d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.244582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43df12-a167-4c59-a7f5-1fb7ace3d938 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.244994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43df12-a167-4c59-a7f5-1fb7ace3d938 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.245227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43df12-a167-4c59-a7f5-1fb7ace3d938 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.246264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43df12-a167-4c59-a7f5-1fb7ace3d938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.247805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43df12-a167-4c59-a7f5-1fb7ace3d938 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.249717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 24/45] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:47.255686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.337002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.368585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87040e-06e5-42fa-81c8-015975ede505 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.403849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0105c7-9ef6-4217-ac34-6198b9882f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.404463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0105c7-9ef6-4217-ac34-6198b9882f2b None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.404744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0105c7-9ef6-4217-ac34-6198b9882f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.405020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0105c7-9ef6-4217-ac34-6198b9882f2b None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.405675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0105c7-9ef6-4217-ac34-6198b9882f2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.407118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0105c7-9ef6-4217-ac34-6198b9882f2b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.438290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 22/46] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:43:47 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:43:49.019321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b648-9ff2-42b5-979e-7702483041bb None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.019848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b648-9ff2-42b5-979e-7702483041bb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.019929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b648-9ff2-42b5-979e-7702483041bb None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.021254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 25/47] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:49.026807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.027284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.027374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.046500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:49.047720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:49.116342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 23/48] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:49.382316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b648-9ff2-42b5-979e-7702483041bb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:49.493939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:49.502820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b648-9ff2-42b5-979e-7702483041bb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:49.536541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132dd2-6075-4662-93a7-5a45415035dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:49.537184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132dd2-6075-4662-93a7-5a45415035dc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.537465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132dd2-6075-4662-93a7-5a45415035dc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.537737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132dd2-6075-4662-93a7-5a45415035dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.549208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00132dd2-6075-4662-93a7-5a45415035dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-00132dd2-6075-4662-93a7-5a45415035dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:49.554337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:49.555288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:43:49.556031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 26/49] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:49.561920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:49.622416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:49.669354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4a2e2-4579-4ac6-9eb7-64349667a92a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:49.688146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002bd9-3e2d-4b4f-bae7-e7823142d6be None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.688436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002bd9-3e2d-4b4f-bae7-e7823142d6be None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.689438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd002bd9-3e2d-4b4f-bae7-e7823142d6be None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.690848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 27/50] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:49.696057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.696335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.696610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.708580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:49.709202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.709488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.709773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.710427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:49.710734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:49.712161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:49.715070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:49.716647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:49.720608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 24/51] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:49.726318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:49.814909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 28/52] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:49.985313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c495de6-1245-42e4-ba76-ce370f29722d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:50.017831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:50.036942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:50.037645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.037915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.038209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.039944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:50.041486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:50.043739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 25/53] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:49 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:50.051789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:50.133888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:50.177623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:50.200557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804d46d-3e5c-403a-a949-eb8a5e7f0753 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:50.247307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:50.247995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.248479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.248758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.250951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:50.252712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:50.254767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 29/54] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:50.260942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:50.265248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c291c2-9ea7-4e3a-9290-5488041e6e78 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:50.311669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:50.312355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.312647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.312928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.314289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:50.315897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:50.317657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 26/55] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:50.324129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:50.411295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:50.412477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883f8f2-eb17-4325-8910-f013982264ed None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:50.462534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:50.463182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.463517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.463797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.464570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.domain.id=default, user.id=f95719d40ecc461fa97c90ba29b8ba24) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:50.464927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:50.466529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d2f30-49dc-49ca-9e2a-a1803b9b2880 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:50.472112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bd69e-b9bb-41ae-9dcb-59523b6a6d76 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:50.474991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 30/56] 10.222.0.66 () {56 vars in 1322 bytes} [Thu Sep 22 14:43:50 2022] GET /identity/v3/role_assignments?user.id=f95719d40ecc461fa97c90ba29b8ba24&scope.domain.id=default&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 530 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:50.513722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6babe61-6103-47e3-b148-aec0109b056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:50.514302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6babe61-6103-47e3-b148-aec0109b056d None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.514588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6babe61-6103-47e3-b148-aec0109b056d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.514863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6babe61-6103-47e3-b148-aec0109b056d None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.515591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6babe61-6103-47e3-b148-aec0109b056d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:50.517650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6babe61-6103-47e3-b148-aec0109b056d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:50.552610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 27/57] 10.222.0.66 () {60 vars in 1140 bytes} [Thu Sep 22 14:43:50 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:43:52.781407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a9bf-2ef4-4573-a6e2-856e36dee614 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:52.781901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a9bf-2ef4-4573-a6e2-856e36dee614 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:52.781901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1a9bf-2ef4-4573-a6e2-856e36dee614 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:52.783275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 31/58] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:52.788654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:52.789116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:52.789209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:52.808117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:52.809346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:52.880065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 28/59] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:53.156256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1a9bf-2ef4-4573-a6e2-856e36dee614 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:53.176364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:53.269001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1a9bf-2ef4-4573-a6e2-856e36dee614 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:53.306269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c2ebe-f5b4-4e2e-ab9f-195733d3fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:53.306906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2ebe-f5b4-4e2e-ab9f-195733d3fa5c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:53.307184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2ebe-f5b4-4e2e-ab9f-195733d3fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:53.307495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2ebe-f5b4-4e2e-ab9f-195733d3fa5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:53.308604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c2ebe-f5b4-4e2e-ab9f-195733d3fa5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:53.310064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c2ebe-f5b4-4e2e-ab9f-195733d3fa5c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:53.311656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 32/60] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:53.318316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:53.358850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:53.419090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccce388-3a1f-42ae-a207-6a05302ee9b9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:53.454202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e5b6-f3f9-42fc-91c1-f0673b06e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:53.454815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e5b6-f3f9-42fc-91c1-f0673b06e3e8 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:53.455089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e5b6-f3f9-42fc-91c1-f0673b06e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:53.455397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e5b6-f3f9-42fc-91c1-f0673b06e3e8 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:53.456052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4e5b6-f3f9-42fc-91c1-f0673b06e3e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:53.457508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4e5b6-f3f9-42fc-91c1-f0673b06e3e8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:53.485953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 29/61] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:43:53 2022] POST /identity/v3/users => generated 275 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:43:57.553252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787db0d7-7430-45c3-9b3c-83b9e1afcf41 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.554208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787db0d7-7430-45c3-9b3c-83b9e1afcf41 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.554707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787db0d7-7430-45c3-9b3c-83b9e1afcf41 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.556374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 33/62] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:57.574459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1db96-407b-4d5a-b834-1bdf5e415b44 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.575533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1db96-407b-4d5a-b834-1bdf5e415b44 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.576012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1db96-407b-4d5a-b834-1bdf5e415b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.604776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5ee1db96-407b-4d5a-b834-1bdf5e415b44 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:57.606401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5ee1db96-407b-4d5a-b834-1bdf5e415b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:57.614351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4841a-86ba-49f8-a911-551260bd6b77 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.614975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4841a-86ba-49f8-a911-551260bd6b77 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.615482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4841a-86ba-49f8-a911-551260bd6b77 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.617076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 34/63] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:57.622900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598ddae-81f9-4f9b-a6c8-289f696663c0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.623509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598ddae-81f9-4f9b-a6c8-289f696663c0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.623987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598ddae-81f9-4f9b-a6c8-289f696663c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.644104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6598ddae-81f9-4f9b-a6c8-289f696663c0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:57.645714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6598ddae-81f9-4f9b-a6c8-289f696663c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:57.711857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 30/64] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:57.713390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b8705-e2ac-4d22-b775-58152c1c20d2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.713676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b8705-e2ac-4d22-b775-58152c1c20d2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.713952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b8705-e2ac-4d22-b775-58152c1c20d2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.715847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 31/65] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:57.722086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.722656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.723122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.740610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 35/66] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:57.744695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:57.746030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:57.769224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:57.789628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861abd1-d0a7-4aec-9f2b-b3e65b89bf67 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.790368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861abd1-d0a7-4aec-9f2b-b3e65b89bf67 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.790850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861abd1-d0a7-4aec-9f2b-b3e65b89bf67 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.792488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 36/67] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:57.798312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:57.798959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:57.799548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:57.821023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:57.822557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:57.851612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:57.854360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 32/68] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:57.943408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 37/69] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:58.120079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.148841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.230665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7a6f-6edc-4d64-8e56-b8519e8f9194 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.267236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d33efd-8be9-416d-999e-a248a84f1a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.267887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33efd-8be9-416d-999e-a248a84f1a7a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.268170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33efd-8be9-416d-999e-a248a84f1a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.268544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d33efd-8be9-416d-999e-a248a84f1a7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.281961 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d33efd-8be9-416d-999e-a248a84f1a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d0d33efd-8be9-416d-999e-a248a84f1a7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:58.288661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:58.289567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:43:58.290320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 33/70] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:58.291539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e542f-8525-45ea-a4b5-3a01e2e557a4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.291539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e542f-8525-45ea-a4b5-3a01e2e557a4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.291792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e542f-8525-45ea-a4b5-3a01e2e557a4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.293036 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 34/71] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:58.293771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e542f-8525-45ea-a4b5-3a01e2e557a4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.299659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa6e0f-b126-4134-bd8e-39d32ce14da7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.311533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.337172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.337817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.338102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.338380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.349218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:58.354164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:58.354996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:43:58.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 38/72] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:43:58.356424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.361046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.407264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e542f-8525-45ea-a4b5-3a01e2e557a4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.441914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.442559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.442844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.443123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.454637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:58.459561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:58.460389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:43:58.461117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 35/73] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:58.462182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599cfb7-f393-4faf-a056-626d692b3806 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.462269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.497569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2cd5d-7e78-447b-8897-c74bc32edb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.498208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2cd5d-7e78-447b-8897-c74bc32edb79 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.498494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2cd5d-7e78-447b-8897-c74bc32edb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.498767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2cd5d-7e78-447b-8897-c74bc32edb79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.510261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2cd5d-7e78-447b-8897-c74bc32edb79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a5c2cd5d-7e78-447b-8897-c74bc32edb79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:58.515309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:58.516175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:43:58.516981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 39/74] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:58.517754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.517986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.518264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.536143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:58.537350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:58.549926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.575688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99f9ed-c83f-490e-a6c1-a8b86cfb9a76 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.599757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.606460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 40/75] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:58.607118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.626503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.627216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.627823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.628155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.628828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:58.629137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:58.631001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:58.646533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 36/76] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:58.646533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.713220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46053-82bd-49e9-9df1-6e176798749c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.752174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.753136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.753563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.754009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.754843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:58.755333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:58.756886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:58.762594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dd5c1-bbb1-420b-b082-b8fcbc2167a2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.767509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 41/77] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:58.772506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.799389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.800010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.800292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.800570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.801295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:58.801530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:58.802904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:58.805786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.815382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 37/78] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:58.816822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.839268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.891713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60209c01-5ab7-40ae-8682-e5d58fc6fcaf None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.932432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.933053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.933335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.933611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.934296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:58.934613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:58.934760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317f42c-807e-4bc4-a268-31aec83fbc8c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.936053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:58.944815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 42/79] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:58.944815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.975323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.975895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.976180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.976459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.993725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:58.995334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.003282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 38/80] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.003282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.060795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.071989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e385fc-970e-4d69-a2de-a488092a1cce None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.091843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.113942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2135fbb-f074-4f38-b231-d3d8ff6a8158 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.114150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.114790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.115075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.115472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.116671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.118183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.119879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 43/81] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.120684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.157639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.158293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.158740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.158845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.160265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.161836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.163517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 39/82] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.164310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.237769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0084250-7fe7-4e41-9793-c3e2946f854f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.272485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.273121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.273223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ac5f-726d-437c-a67e-67618f24b42f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.273404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.273678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.274805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.276328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.277901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 44/83] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.278672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.298949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.309987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.310632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.310912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.311191 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.312489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.314000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.315620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 40/84] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.316391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.337282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.383188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc74da-ec5d-488c-960b-4d9c00c3a32a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.417480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.417920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04c65d-b1a4-49cb-87ea-fee86cb3b15f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.418092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.418377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.418653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.451314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.451915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.452196 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.452467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.489183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.490622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.510594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.521133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.522604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.545590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.590554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 45/85] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:43:59.591475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.620205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 41/86] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:43:59.621044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.692971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e4e9b-2f39-4011-9e58-49b084951a75 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.717737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.727242 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d6509-2c92-4784-8ed0-1662888621f4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.728356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.729004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.729288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.730232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.755714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.764890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.765512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.765792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.766069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.767314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.768756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.790070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 42/87] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:43:59 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:59.790911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.825802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.827810 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.904950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52aa36a-4b59-4c0b-a311-19e13bbda055 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.947345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 46/88] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:43:59.947345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.949112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.949766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.950054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.950331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.971802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.013619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.035158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.036678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.075003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6596ae9-0a0d-4c93-bf25-91eede5bea68 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.122461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.123295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.123588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.123872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.126492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.128108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.129969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 47/89] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.131093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.160461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 43/90] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:00.161626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.231453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.261193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddadb6-4dcb-451c-905a-434189e07619 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.291118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.294505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819cba5-08c1-438f-b9cc-511277ddb7ba None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.306298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.307231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.307712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.308159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.309629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.311657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.314813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 48/91] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.315947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.342976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.343929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.345027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.345493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.346985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.348662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.350970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 44/92] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.352399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.454316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b3d4c-9a8b-4d8c-bf8c-6e06bc9d2bb7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.496040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.496354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b0ea-0ed1-4082-8f73-93a5ed5f5847 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.497167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.497659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.498163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.499233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=a775d9fba1794acc91783971d2655e6b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:00.499559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.501176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.510716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 49/93] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/role_assignments?user.id=a775d9fba1794acc91783971d2655e6b&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.511674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.538932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.576910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.577955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.578449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.579132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] PATH_INFO: `/v3/projects/60576692b576463b8349a055e68720bf` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.582101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] RBAC: Authorizing `identity:get_project(project_id=60576692b576463b8349a055e68720bf)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.588362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.590386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 45/94] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/projects/60576692b576463b8349a055e68720bf => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.590942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.623729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.719817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e68467-1af9-4d3b-b951-d4a074f16947 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.760649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa642640-e5fd-49ac-a717-6659e3bbea42 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.801695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.802312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.802597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.802883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.803653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=f95719d40ecc461fa97c90ba29b8ba24) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:00.803966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.805433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.819100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 50/95] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/role_assignments?user.id=f95719d40ecc461fa97c90ba29b8ba24&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.819840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.834603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.835162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.835485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.835758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.836805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] RBAC: Adding query filter params (role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=a775d9fba1794acc91783971d2655e6b) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:00.837311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.839450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c8617-2870-49db-b84d-1fee569496a6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.860903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 46/96] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/role_assignments?user.id=a775d9fba1794acc91783971d2655e6b&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.972588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb00c8-c12f-43c4-918c-7c22be34ca36 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:01.004982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:01.036150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:01.036798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:01.037085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:01.037369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.038127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=60576692b576463b8349a055e68720bf, user.id=a775d9fba1794acc91783971d2655e6b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:01.038451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.040032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2f96-a8b1-41b8-957c-64b737ba8fd4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.049740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 51/97] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/role_assignments?user.id=a775d9fba1794acc91783971d2655e6b&scope.project.id=60576692b576463b8349a055e68720bf&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:03.477809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1f424-4cc3-47e6-a275-a48bf96159bd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.478678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1f424-4cc3-47e6-a275-a48bf96159bd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.479154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1f424-4cc3-47e6-a275-a48bf96159bd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.480751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 47/98] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.482558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9883f-86fc-418d-a2b1-ea432b6943c3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.483402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9883f-86fc-418d-a2b1-ea432b6943c3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.483890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9883f-86fc-418d-a2b1-ea432b6943c3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.486035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 52/99] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.492159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ffe27-b69b-49e5-93af-34b3de888893 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.492852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ffe27-b69b-49e5-93af-34b3de888893 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.493323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ffe27-b69b-49e5-93af-34b3de888893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.499548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f582f-cd4e-41e5-a480-cd0e2fe67e4f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.501534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f582f-cd4e-41e5-a480-cd0e2fe67e4f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.502930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f582f-cd4e-41e5-a480-cd0e2fe67e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.523855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f08ffe27-b69b-49e5-93af-34b3de888893 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:03.528604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f08ffe27-b69b-49e5-93af-34b3de888893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:03.543972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-798f582f-cd4e-41e5-a480-cd0e2fe67e4f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:03.548341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-798f582f-cd4e-41e5-a480-cd0e2fe67e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:03.635332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:03.697141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 48/100] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:03.698066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 53/101] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:03.698851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f20c87-7fc0-45a2-bfda-9f0c3a3be1ba None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.699132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f20c87-7fc0-45a2-bfda-9f0c3a3be1ba None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.704285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f20c87-7fc0-45a2-bfda-9f0c3a3be1ba None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.705969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 49/102] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.713138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237458d-f412-45e7-b1c8-406f1c66f910 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.713431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237458d-f412-45e7-b1c8-406f1c66f910 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.713702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237458d-f412-45e7-b1c8-406f1c66f910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.756327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e237458d-f412-45e7-b1c8-406f1c66f910 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:03.757466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e237458d-f412-45e7-b1c8-406f1c66f910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:03.834939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ffa66-d598-4146-a086-4250c1e4d24a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.835590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ffa66-d598-4146-a086-4250c1e4d24a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.836058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ffa66-d598-4146-a086-4250c1e4d24a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.838040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 50/103] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.844879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614f3e-1d2e-4fde-93b6-160e4770d6ad None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.845505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614f3e-1d2e-4fde-93b6-160e4770d6ad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.846009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614f3e-1d2e-4fde-93b6-160e4770d6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.859249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 54/104] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:03.884639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d5614f3e-1d2e-4fde-93b6-160e4770d6ad None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:03.886479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d5614f3e-1d2e-4fde-93b6-160e4770d6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:03.990862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 51/105] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:04.029808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690133-5d47-4e6a-9f95-6b5af664c1d3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.030078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690133-5d47-4e6a-9f95-6b5af664c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.030298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12690133-5d47-4e6a-9f95-6b5af664c1d3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.031877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 55/106] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:04.037036 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.037329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.037608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.065798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:04.067142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:04.121325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12690133-5d47-4e6a-9f95-6b5af664c1d3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.159389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 52/107] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:04.164369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.224598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.259580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.321850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12690133-5d47-4e6a-9f95-6b5af664c1d3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.353461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c2f3d-ed49-4b24-8b61-d69d740486c8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.373516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425eba68-06b5-4a50-8b9d-662a6c94bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.374157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eba68-06b5-4a50-8b9d-662a6c94bed8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.374440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eba68-06b5-4a50-8b9d-662a6c94bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.374719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425eba68-06b5-4a50-8b9d-662a6c94bed8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.376411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425eba68-06b5-4a50-8b9d-662a6c94bed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:04.377878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425eba68-06b5-4a50-8b9d-662a6c94bed8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:04.379500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 56/108] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:04.380784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ee63a-110e-4cc1-b104-9b3c97bf5995 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.380990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ee63a-110e-4cc1-b104-9b3c97bf5995 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.381255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ee63a-110e-4cc1-b104-9b3c97bf5995 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.382638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 57/109] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:04.385585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ee63a-110e-4cc1-b104-9b3c97bf5995 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.399025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.399985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.400457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.401018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.412178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.417510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.418363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:04.419107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 53/110] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:04.419765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.502168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ee63a-110e-4cc1-b104-9b3c97bf5995 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.513824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.528563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.532227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8ab5e-6da2-4dc8-9937-32bb800bfe3e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.539868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.540688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.540972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.541249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.552032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.556894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.557739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 58/111] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:04.559951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.569662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26d90c-99bd-4d25-bc80-8f2603176725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.570304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26d90c-99bd-4d25-bc80-8f2603176725 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.570583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26d90c-99bd-4d25-bc80-8f2603176725 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.570854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26d90c-99bd-4d25-bc80-8f2603176725 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.580946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26d90c-99bd-4d25-bc80-8f2603176725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2d26d90c-99bd-4d25-bc80-8f2603176725 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.585712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.586562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 54/112] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:04.592495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.593157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.593157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.606312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:04.607546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:04.659699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639776-131d-485d-b46f-ab9b35c8caaf None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.677298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 55/113] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:04.678172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.698044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.698658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.698940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.699242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.699944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:04.701419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:04.731741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 59/114] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:44:04 2022] POST /identity/v3/projects => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:04.732581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.764582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.846750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.864713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da682d34-035e-4d73-a60c-4d0f0265498f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.868667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54070f-ca51-4b92-9006-23b3eeef5327 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.908153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.908816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.909101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.909377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.910057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:04.910367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:04.912602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:04.918430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.919993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.920462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.921211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.928171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 56/115] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:04.928171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.943006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.949080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.949944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:04.950693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 60/116] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:04.952044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.057378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e00155-d4e3-459f-8189-3022e40b80ac None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.059223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.085775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78406680-61dd-4a4c-b730-832bf2c36e60 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.097368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.098268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.098758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.099040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.099749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:05.100058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.101479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.111469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 57/117] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.112159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.129750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.130281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.130911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.131271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.131649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.132264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:05.132578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.134244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.145035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 61/118] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.146586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.228162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2e6b6-f0fd-4294-8355-3b7cfa8475f8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.266765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.267442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.267570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f473e97-f4c0-4f77-b0f8-82cc6ca91606 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.267756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.267999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] PATH_INFO: `/v3/users/f8d77a30d47a4b6ba01be094a6831183` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.282494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] RBAC: Authorizing `identity:get_user(user_id=f8d77a30d47a4b6ba01be094a6831183)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.284009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.290056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 58/119] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/f8d77a30d47a4b6ba01be094a6831183 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.290056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.313502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.314141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.314431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.314710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.325514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:05.331025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:05.331923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:05.332734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 62/120] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:05.334262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.338920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.367400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.409132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87b45e-f277-4d8f-a261-63153eec849a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.445631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.446244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.446686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.446790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.447484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:05.447796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.449185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.449547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5dd1-7403-4e66-acb8-6fd3e396fc57 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.458853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 59/121] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.459752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.502323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.503094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.503410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.503691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.504862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.506439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.513016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 63/122] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.513016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.577048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b910d-a67c-44f7-bc59-1ea9e07fec18 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.588657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.610743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.618629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.619544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.620180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.620612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.622104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.623025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404a497-ddaf-471e-b93a-7a51ee525aa2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.623886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.626913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 60/123] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.627864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.660285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.660902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.661183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.661462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.742351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a8a0f-0127-4eb2-815e-dcd0d9d59f84 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.742560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.744011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.782679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.783325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.783631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.783908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.784587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:05.784893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.786287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.794905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 61/124] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.800395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.820647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.858433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 64/125] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:05.859675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.870406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.916892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef2d22-b4f0-47ae-8863-571aabf23903 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.958341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.958990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.959290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.959568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.960719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.962693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.964339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 62/126] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.965119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.976129 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83bd7d-9d3e-46d3-8bcd-9381201b3682 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.013628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.014274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.014558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.014839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.016231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.017535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.019215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 65/127] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.019997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.073176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.097547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3754c8-ce41-412c-a3bc-272d5ff6d437 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.133640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.142024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.142671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.143060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.143376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.144619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.146195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.152827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 63/128] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.152827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.153029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7c141-f578-4ccf-bbe5-c8aee21f7f0a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.186740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.187410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.187695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.187972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.189875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.191408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.193071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 66/129] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.193857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.273702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1a8be-0e74-43a4-ac6f-31ab9ed8ba7c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.321593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.322246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.322532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.322823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.324045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.325514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.327369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 64/130] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.328231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d51ba-6c86-4407-87dd-57bf4fd50d23 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.332888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.339594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.370586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.371271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.371556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.371887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.373633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.379618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.379618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 67/131] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.379618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.449604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.496987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c634-c849-4ad5-bb1c-ed75fb84965f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.504071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2373a5-cf34-4209-bdbb-f32e83c4b6b7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.538783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.539510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.539793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.540071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/a775d9fba1794acc91783971d2655e6b/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.541859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.542230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=90c1eb704e8b41d8a50cece9255ebf24, user_id=a775d9fba1794acc91783971d2655e6b, role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.542528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.542811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.543088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/a775d9fba1794acc91783971d2655e6b/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.543764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.545121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] RBAC: Authorizing `identity:create_grant(project_id=90c1eb704e8b41d8a50cece9255ebf24, user_id=a775d9fba1794acc91783971d2655e6b, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.546733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.571238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 65/132] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:06 2022] PUT /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/a775d9fba1794acc91783971d2655e6b/roles/e1b5b7a11157451ab80ef767124856f0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:06.571462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 68/133] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:06 2022] PUT /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/a775d9fba1794acc91783971d2655e6b/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:06.571572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.571642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.659780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.784274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501d029-0c84-4138-85b7-92a6f2133bbb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.806742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd1ab6-17b9-43d8-8212-c9a130ee04eb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.814095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.825695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.826345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.826676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.826906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] PATH_INFO: `/v3/projects/60576692b576463b8349a055e68720bf` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.829102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] RBAC: Authorizing `identity:get_project(project_id=60576692b576463b8349a055e68720bf)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.830598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.836661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 66/134] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/projects/60576692b576463b8349a055e68720bf => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.837396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.853953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.854574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.854858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.855137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.855917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] RBAC: Adding query filter params (role.id=d92a531856fd46cc87fffc7a3af54ac7, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=f8d77a30d47a4b6ba01be094a6831183) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:06.856231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.857772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.864725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 69/135] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/role_assignments?user.id=f8d77a30d47a4b6ba01be094a6831183&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=d92a531856fd46cc87fffc7a3af54ac7 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.865545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.952696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f1b55-3e22-46cf-b8f1-f8967eb181af None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.991080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2e3c3-a295-4568-b0fa-25dfde1d3309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.991817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2e3c3-a295-4568-b0fa-25dfde1d3309 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.992098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2e3c3-a295-4568-b0fa-25dfde1d3309 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.992379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2e3c3-a295-4568-b0fa-25dfde1d3309 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/f95719d40ecc461fa97c90ba29b8ba24/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.994396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2e3c3-a295-4568-b0fa-25dfde1d3309 None admin] RBAC: Authorizing `identity:create_grant(project_id=90c1eb704e8b41d8a50cece9255ebf24, user_id=f95719d40ecc461fa97c90ba29b8ba24, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.995923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2e3c3-a295-4568-b0fa-25dfde1d3309 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.011051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 67/136] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:06 2022] PUT /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/f95719d40ecc461fa97c90ba29b8ba24/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:07.070524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f31438-7d91-46d8-846d-464469390c2e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.070904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f31438-7d91-46d8-846d-464469390c2e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.071288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f31438-7d91-46d8-846d-464469390c2e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.074386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 68/137] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:07.085281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.085725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.086119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.099947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:07.104595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92bb6a-b0a1-412d-9d09-a221b36a5ff5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.126758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:07.128039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:07.181625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.182320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.182607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.182891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] PATH_INFO: `/v3/projects/60576692b576463b8349a055e68720bf/users/a775d9fba1794acc91783971d2655e6b/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.188717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] RBAC: Authorizing `identity:create_grant(project_id=60576692b576463b8349a055e68720bf, user_id=a775d9fba1794acc91783971d2655e6b, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.190234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.212683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:07.218078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 70/138] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:06 2022] PUT /identity/v3/projects/60576692b576463b8349a055e68720bf/users/a775d9fba1794acc91783971d2655e6b/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:07.247971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 69/139] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 22 14:44:07.618642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.746234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98b1ef-4a71-47ec-9d42-2981e77cca1e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.795939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da29e3e-ea48-43c2-8789-ce1db48af46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.796591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da29e3e-ea48-43c2-8789-ce1db48af46f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.796874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da29e3e-ea48-43c2-8789-ce1db48af46f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.797150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da29e3e-ea48-43c2-8789-ce1db48af46f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.798853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da29e3e-ea48-43c2-8789-ce1db48af46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.800346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da29e3e-ea48-43c2-8789-ce1db48af46f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.802071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 71/140] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:07.808111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.945257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e24833-707b-46aa-89ad-a388764d16a8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.989514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa79869-b407-4f8c-86d0-e059fd0bb6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.990205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa79869-b407-4f8c-86d0-e059fd0bb6e8 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.990485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa79869-b407-4f8c-86d0-e059fd0bb6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.990759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa79869-b407-4f8c-86d0-e059fd0bb6e8 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.991733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa79869-b407-4f8c-86d0-e059fd0bb6e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.993759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa79869-b407-4f8c-86d0-e059fd0bb6e8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:08.020399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 70/141] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:07 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:09.573683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36745c-dcf7-4ea3-b26c-85dcab4f8bce None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.574221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36745c-dcf7-4ea3-b26c-85dcab4f8bce None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.574221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36745c-dcf7-4ea3-b26c-85dcab4f8bce None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.575611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 72/142] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:09.581404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317661d-d570-4df3-b0b4-ae7d725c2d05 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.581869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317661d-d570-4df3-b0b4-ae7d725c2d05 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.581948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317661d-d570-4df3-b0b4-ae7d725c2d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.605002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2317661d-d570-4df3-b0b4-ae7d725c2d05 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:09.606207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2317661d-d570-4df3-b0b4-ae7d725c2d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:09.655128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:09.697895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 71/143] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:10.144586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181cf91-a320-4941-b856-25951d0f3cca None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.144984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181cf91-a320-4941-b856-25951d0f3cca None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.145278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181cf91-a320-4941-b856-25951d0f3cca None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.146861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 73/144] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:10.152776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac36f67-9caf-4ddd-ba1d-171755e6fe7e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.153073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac36f67-9caf-4ddd-ba1d-171755e6fe7e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.153355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac36f67-9caf-4ddd-ba1d-171755e6fe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.183834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6ac36f67-9caf-4ddd-ba1d-171755e6fe7e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:10.185979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6ac36f67-9caf-4ddd-ba1d-171755e6fe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:10.274706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 72/145] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:10.307569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5a5e7-bd4e-425a-ae84-02e6ab16ef66 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.307890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5a5e7-bd4e-425a-ae84-02e6ab16ef66 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.308136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5a5e7-bd4e-425a-ae84-02e6ab16ef66 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.309516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 74/146] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:10.315979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.316265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.316541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.318226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5a5e7-bd4e-425a-ae84-02e6ab16ef66 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.340285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:10.341580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:10.366443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.423476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 73/147] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:10.495449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5a5e7-bd4e-425a-ae84-02e6ab16ef66 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.549067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.549718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.550000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.550277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.566782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:10.571767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:10.572643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:10.573410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 75/148] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:10.579057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.646406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.739431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688f659-f22e-4856-8b74-c7e4acde8906 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.775611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.778125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.778556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.778966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.780214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:10.780894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:10.783410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab90759-e96b-4701-9940-177b06d71897 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:10.792368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 74/149] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:10.798625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.834010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532aee-3d45-432e-9e4a-763ea3f33294 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.834782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532aee-3d45-432e-9e4a-763ea3f33294 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.835258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532aee-3d45-432e-9e4a-763ea3f33294 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.837414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 75/150] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:10.844727 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.845217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.845739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.869978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:10.871985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:10.954497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 76/151] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:10.955754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.995683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.030393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.068348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c1c1d-06a2-42dd-a579-679d2752a6cd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.071835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9dde81-505b-4d1c-ba6d-15d2cf7b27a4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.111830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.112807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.113387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.113814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.125004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:11.130387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:11.131344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:11.132082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:11.134664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 77/152] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:11.135724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.171959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.173174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.173461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.173737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.180694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.181737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.187837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 76/153] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.187837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.256547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.278259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61bb1-12fb-4220-9da2-2e830a81b7de None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.333135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.334061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.334568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.334973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.348430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:11.353831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:11.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:11.355787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:11.357967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 78/154] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:11.358953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.419332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd8bcd-9db0-4ee7-ad80-7853d7ba8df2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.476835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6eee6b-a3c8-413a-95ff-4c7c24fc0189 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.509797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.510354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.510636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.510912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.511799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:11.512070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.513631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f803ce-dcdd-420d-8fb1-9eb52d08f99f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.527099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.528021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.528492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.528894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.537595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 77/155] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.539300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27f0a-16e4-42ed-a99b-4065d41696a8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.544314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27f0a-16e4-42ed-a99b-4065d41696a8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.544815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd27f0a-16e4-42ed-a99b-4065d41696a8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.546649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 78/156] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:11.548351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd27f0a-16e4-42ed-a99b-4065d41696a8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.560256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.620441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.629257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.631056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.767129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 79/157] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:11.768013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.839617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.869919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4d51a-bccd-4f63-94b9-e26bb34a5c14 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.908537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.909198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.909476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.909747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.910420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:11.910724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.912247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.924017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd27f0a-16e4-42ed-a99b-4065d41696a8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.926863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 80/158] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.927876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.019404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eb453-045f-4440-91a8-90a75e1806cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.020031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eb453-045f-4440-91a8-90a75e1806cf None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.020312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eb453-045f-4440-91a8-90a75e1806cf None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.020577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eb453-045f-4440-91a8-90a75e1806cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.040682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e026d7e-541e-4766-ad96-015ccb7b555b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.055858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eb453-045f-4440-91a8-90a75e1806cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-164eb453-045f-4440-91a8-90a75e1806cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:12.061278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:12.062192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:12.063598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 79/159] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 514 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:12.064705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.064705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.064922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.065773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.078213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.078905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.079175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.079486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.080641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.087259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.087259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 81/160] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.089337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.104864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:12.106586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:12.200607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2883f-7de6-4e14-8744-b0c039d49843 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.238708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.239376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.239657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.239938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.241056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.242252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 80/161] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:12.243314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.247676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.249526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 82/162] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.251337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.270746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.337559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.454981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5c14-abbb-47eb-becd-caaba48132bc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.490838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765e396-ffda-461d-989e-f4cdcab89f68 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.520566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.521229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.521448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.521715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.522373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:12.522672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.524071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.540313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 83/163] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.541166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.559702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.560343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.560612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.560879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] PATH_INFO: `/v3/users/f8d77a30d47a4b6ba01be094a6831183` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.562885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] RBAC: Authorizing `identity:get_user(user_id=f8d77a30d47a4b6ba01be094a6831183)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.564445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.566033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 81/164] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/users/f8d77a30d47a4b6ba01be094a6831183 => generated 289 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.566794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.686510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.690696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb9289-8958-42f0-8929-ab7d5dd75370 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.693378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.718471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998ca65-2f81-4a14-975a-d273587afd29 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.738075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.738903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.739615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.739894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.756972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.757890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.758665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.759323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.760400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] RBAC: Adding query filter params (role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=a775d9fba1794acc91783971d2655e6b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:12.760804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.762847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28353fa9-4630-4d5b-b0d7-f0c25036b895 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.772080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 82/165] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/role_assignments?user.id=a775d9fba1794acc91783971d2655e6b&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.773692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf475c5-580d-4366-a369-294d13059412 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.773975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf475c5-580d-4366-a369-294d13059412 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.774423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf475c5-580d-4366-a369-294d13059412 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.776445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 83/166] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:12.777253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf475c5-580d-4366-a369-294d13059412 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.828888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.830437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.926065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf475c5-580d-4366-a369-294d13059412 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.955944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 84/167] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:12.956829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.959361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.974060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.975144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.979982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.980473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] PATH_INFO: `/v3/users/a775d9fba1794acc91783971d2655e6b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.982049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] RBAC: Authorizing `identity:get_user(user_id=a775d9fba1794acc91783971d2655e6b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.983800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.985795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 84/168] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/users/a775d9fba1794acc91783971d2655e6b => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.986800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.010343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.087746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441d95-2354-46a3-a4d6-0e593d4c1c8d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.139312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc55b0-4300-40a5-8a6f-e0bfc1a55d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.140009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc55b0-4300-40a5-8a6f-e0bfc1a55d48 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.140288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc55b0-4300-40a5-8a6f-e0bfc1a55d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.140685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc55b0-4300-40a5-8a6f-e0bfc1a55d48 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.142699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc55b0-4300-40a5-8a6f-e0bfc1a55d48 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.144587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc55b0-4300-40a5-8a6f-e0bfc1a55d48 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.146501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 85/169] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.147618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c83dc3-2430-48b6-b400-b7d8c31d0bbf None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.147864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.148171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.148453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.177641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:13.179150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:13.194806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.195480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.195816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.196193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.209658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:13.215772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:13.216646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 85/170] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:13.218460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.276159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 86/171] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:13.277122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.279954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.316515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.366015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a644bad-921a-4260-b329-dbed970d1889 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.394881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5935f-0963-465c-b197-0a9f985e7875 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.408238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.408880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.409203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.409432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.410651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.412229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.419277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 86/172] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.419277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.435141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b756-ec0b-453b-8041-a5e4573536d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.435792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b756-ec0b-453b-8041-a5e4573536d8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.436241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b756-ec0b-453b-8041-a5e4573536d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.436517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b756-ec0b-453b-8041-a5e4573536d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.518478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.521067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b756-ec0b-453b-8041-a5e4573536d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.523935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b756-ec0b-453b-8041-a5e4573536d8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.539287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fa801-f7eb-4bda-b599-8678248889dd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.581631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.582304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.582569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.582837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/f8d77a30d47a4b6ba01be094a6831183/roles/d92a531856fd46cc87fffc7a3af54ac7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.585133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] RBAC: Authorizing `identity:create_grant(project_id=90c1eb704e8b41d8a50cece9255ebf24, user_id=f8d77a30d47a4b6ba01be094a6831183, role_id=d92a531856fd46cc87fffc7a3af54ac7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.586834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.591747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.605789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 87/173] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:13 2022] PUT /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/users/f8d77a30d47a4b6ba01be094a6831183/roles/d92a531856fd46cc87fffc7a3af54ac7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:13.606792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 87/174] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a619-ad54-4b39-b556-7399b6ad50cc None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a619-ad54-4b39-b556-7399b6ad50cc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a619-ad54-4b39-b556-7399b6ad50cc None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 88/175] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc195be9-037d-4f7e-9d6d-0e5e5ab0cccc None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.666723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc195be9-037d-4f7e-9d6d-0e5e5ab0cccc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.667338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc195be9-037d-4f7e-9d6d-0e5e5ab0cccc None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.667987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 89/176] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:13.668704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc195be9-037d-4f7e-9d6d-0e5e5ab0cccc None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.755749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74161e3-e097-496d-8e99-c0f411ff924f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.794444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.795178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.795555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.795830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.796497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:13.796803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.798753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.809902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 88/177] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.810762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.818198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.824821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc195be9-037d-4f7e-9d6d-0e5e5ab0cccc None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.866462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.867141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.867452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.867640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.868383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=a775d9fba1794acc91783971d2655e6b) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:13.868694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.870617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.878202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 90/178] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/role_assignments?user.id=a775d9fba1794acc91783971d2655e6b&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.879127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.893271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.982231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274ecb3-0251-4e39-9be4-7bd50ce8d655 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.044140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61717249-b72b-47e4-b195-8358a2f63ad7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.046425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98512e55-11ec-4a2e-b480-e2282d9e2d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.047929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98512e55-11ec-4a2e-b480-e2282d9e2d47 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.048424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98512e55-11ec-4a2e-b480-e2282d9e2d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.048913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98512e55-11ec-4a2e-b480-e2282d9e2d47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.061898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98512e55-11ec-4a2e-b480-e2282d9e2d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-98512e55-11ec-4a2e-b480-e2282d9e2d47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:14.068006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:14.068942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:14.069725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:14.072507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 89/179] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:14.074654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.075306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.075830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.096046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d0c7f-2240-4fe6-a056-675a8f80eb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.096722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d0c7f-2240-4fe6-a056-675a8f80eb25 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.097005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d0c7f-2240-4fe6-a056-675a8f80eb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.097306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d0c7f-2240-4fe6-a056-675a8f80eb25 None admin] PATH_INFO: `/v3/projects/60576692b576463b8349a055e68720bf` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.098501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d0c7f-2240-4fe6-a056-675a8f80eb25 None admin] RBAC: Authorizing `identity:get_project(project_id=60576692b576463b8349a055e68720bf)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.099183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:14.100058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d0c7f-2240-4fe6-a056-675a8f80eb25 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.100924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:14.101814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 91/180] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/projects/60576692b576463b8349a055e68720bf => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.107666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.108280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.108775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.135831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:14.135831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:14.170720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.210449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 90/181] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:14.211630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.229160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 92/182] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:14.230559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.300050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.364930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbff313-c9c9-429f-b8e8-fe9b21f6fab6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.416582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.417213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.417490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.417759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.418431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:14.418733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.420945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.436245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 93/183] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.438529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.443601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07812bbd-be96-48fe-87ba-88651251da74 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.479076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.517240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.517920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.518199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.518470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] PATH_INFO: `/v3/users/f95719d40ecc461fa97c90ba29b8ba24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.519636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] RBAC: Authorizing `identity:get_user(user_id=f95719d40ecc461fa97c90ba29b8ba24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.521096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.522695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 91/184] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/users/f95719d40ecc461fa97c90ba29b8ba24 => generated 247 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.523571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.565491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ff985-7198-4f70-ab12-711b840a692e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.610141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.610876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.611157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.611481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.612282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=60576692b576463b8349a055e68720bf, user.id=a775d9fba1794acc91783971d2655e6b) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:14.612589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.614219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.620935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 94/185] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/role_assignments?user.id=a775d9fba1794acc91783971d2655e6b&scope.project.id=60576692b576463b8349a055e68720bf&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.621834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.695021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d0e81-fd9e-40e4-b1d3-32a7e0a1f89e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.714642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.740518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.741182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.741467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.741744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] PATH_INFO: `/v3/users/84d554f2aeb4485e9b94ca112d0934d6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.753987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.759121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] RBAC: Authorizing `identity:get_user(user_id=84d554f2aeb4485e9b94ca112d0934d6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.760785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.762977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 92/186] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/users/84d554f2aeb4485e9b94ca112d0934d6 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.763970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.773990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15540e47-b14a-4ccd-a8ac-cf82bd227333 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.824645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.825610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.825885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.826159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.902337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1190a6-9008-40f8-a393-681534ddba08 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.907514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.909001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.945099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.945733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.946013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.946285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.947443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.949585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.955845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 93/187] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.955845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.005953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.017099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.032346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 95/188] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:15.033493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.092941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf180dbf-af5d-4897-8a68-e4225ad8e01c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.131312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.131952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.132229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.132500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.145559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:15.150607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:15.151567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:15.152563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 94/189] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:15.157336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.177265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5d6fb-6d55-419b-bb2f-daa9fd497fcc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.226570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.227601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.228081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.228529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.291042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6a539-4745-4994-a9da-093fd2489336 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.292904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.308336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.327148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.329498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.343750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.345070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.345352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.345620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.346288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.347803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.386412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 95/190] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:15 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:15.386412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.505487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 96/191] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:15.511951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.563613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fd6c0-9a82-41eb-b819-37665795f639 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.618223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.619234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.619694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.620165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.621754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.623774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.626319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 96/192] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.631366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.637534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.659442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec808417-6923-4ff4-9d6c-18dc7ae67a81 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.678476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.714888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.715713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.716059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.716403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.717816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:15.718207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.719953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.732202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 97/193] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.732974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.789890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cb330-75d6-4983-bf38-0cbcfda21522 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.845819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.846480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.846766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.847033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.848303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.849792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.852976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 97/194] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.853947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.880778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e1325-e45f-4a33-bfa0-5506095c7565 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.920252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.921180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.921180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.921447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.922890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=f95719d40ecc461fa97c90ba29b8ba24) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:15.923239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.926330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.934216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 98/195] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/role_assignments?user.id=f95719d40ecc461fa97c90ba29b8ba24&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.935081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.965884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.051380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.074667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe9f5-eec8-4bfc-b186-2f5b75da1f95 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.142840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625e83e-a527-424e-8f89-a2de2111af5b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.147185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd555e-0e35-4882-af46-cb95016a2624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.147923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd555e-0e35-4882-af46-cb95016a2624 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.148207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd555e-0e35-4882-af46-cb95016a2624 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.148534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd555e-0e35-4882-af46-cb95016a2624 None admin] PATH_INFO: `/v3/users/84d554f2aeb4485e9b94ca112d0934d6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.152024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd555e-0e35-4882-af46-cb95016a2624 None admin] RBAC: Authorizing `identity:get_user(user_id=84d554f2aeb4485e9b94ca112d0934d6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.153640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd555e-0e35-4882-af46-cb95016a2624 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.155699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 98/196] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/users/84d554f2aeb4485e9b94ca112d0934d6 => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.157476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788931a-538b-491a-bb2b-35658a4ce111 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.157759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788931a-538b-491a-bb2b-35658a4ce111 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.158034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788931a-538b-491a-bb2b-35658a4ce111 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.167660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 99/197] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:16.168558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788931a-538b-491a-bb2b-35658a4ce111 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.196095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.196733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.197016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.197293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.198048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=84d554f2aeb4485e9b94ca112d0934d6) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:16.198359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.199981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645725ca-f8fa-4505-9621-93750b5ad24f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.207023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 99/198] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/role_assignments?user.id=84d554f2aeb4485e9b94ca112d0934d6&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.208479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.208770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.209089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.231791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:16.233097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:16.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 100/199] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:16.382061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788931a-538b-491a-bb2b-35658a4ce111 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.429738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.430592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.431016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.431475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.463966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.506112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.507642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.614375 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 100/200] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:16.620256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.636534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.710476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.753553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d7dd-bea7-44b8-b56f-7a193fccfbbf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.791311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.791955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.792284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.792562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.793750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.795226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.797232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 101/201] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.802841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.819896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.882975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15df8c-399d-4894-acc3-cd292a6573ea None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.889221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.916847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b05c1a-096a-4e13-975a-1a6e7e8a1bbc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.928348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.929555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.930077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.930685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.943432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:16.949256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:16.950164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:16.950948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:16.954294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 101/202] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:16.965130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.965830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.966465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.966744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.967021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.971372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] RBAC: Adding query filter params (role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=84d554f2aeb4485e9b94ca112d0934d6) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:16.971372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.971372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.976887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 102/203] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/role_assignments?user.id=84d554f2aeb4485e9b94ca112d0934d6&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:17.149986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c1d15-83cd-412b-be7f-51a545f9f449 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.175490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.194049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.195002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.195532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.196024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.196904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:17.197411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.199062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.212314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 102/204] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:17.218455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.332315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046a8d-f61b-4fab-810a-eadac6d5cf34 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.369915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.370565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.370843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.371121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] PATH_INFO: `/v3/users/f8d77a30d47a4b6ba01be094a6831183` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.372315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] RBAC: Authorizing `identity:get_user(user_id=f8d77a30d47a4b6ba01be094a6831183)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.373788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.376177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 103/205] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/users/f8d77a30d47a4b6ba01be094a6831183 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:17.382565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.534164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91710fdb-f120-4164-a6bb-00d134ec81dd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.586728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.587388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.587672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.587947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.661367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.682552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.684066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.819784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 103/206] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:17.825115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.855570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.991445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe85ea-7521-450b-9e23-bb4b2298753e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:18.048985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f839e44-7270-4f4a-8e86-728f70e00928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:18.050008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839e44-7270-4f4a-8e86-728f70e00928 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.050499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839e44-7270-4f4a-8e86-728f70e00928 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.051005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f839e44-7270-4f4a-8e86-728f70e00928 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.054695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839e44-7270-4f4a-8e86-728f70e00928 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:18.056401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f839e44-7270-4f4a-8e86-728f70e00928 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:18.058297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 104/207] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:18.064298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:18.176498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:18.196203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2418a-ed9a-4869-97d4-e31ef433dfdd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:18.248148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:18.248772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.249044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.249313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.250084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] RBAC: Adding query filter params (role.id=d92a531856fd46cc87fffc7a3af54ac7, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24, user.id=f8d77a30d47a4b6ba01be094a6831183) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:18.250397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:18.251899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2099ba-6ece-4efc-9451-086871d5902a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:18.263850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 104/208] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:18 2022] GET /identity/v3/role_assignments?user.id=f8d77a30d47a4b6ba01be094a6831183&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=d92a531856fd46cc87fffc7a3af54ac7 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:18.644023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5cda58-7657-4913-9110-e951fc4f330a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.644723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5cda58-7657-4913-9110-e951fc4f330a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.645213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5cda58-7657-4913-9110-e951fc4f330a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.647308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 105/209] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:18.652879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd115ad-1112-4691-a4d2-adbab0d7ad19 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.653496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd115ad-1112-4691-a4d2-adbab0d7ad19 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.653970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd115ad-1112-4691-a4d2-adbab0d7ad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.679976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0dd115ad-1112-4691-a4d2-adbab0d7ad19 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:18.681537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0dd115ad-1112-4691-a4d2-adbab0d7ad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:18.755968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 105/210] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:18.780287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ad2aa-72dc-4f99-9639-e7f30c72299b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.780594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ad2aa-72dc-4f99-9639-e7f30c72299b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.780844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ad2aa-72dc-4f99-9639-e7f30c72299b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.782566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 106/211] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:18.788338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb5f2d-e96e-4a78-97c6-3c500df138a4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.788630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb5f2d-e96e-4a78-97c6-3c500df138a4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.788896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb5f2d-e96e-4a78-97c6-3c500df138a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.812128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4cdb5f2d-e96e-4a78-97c6-3c500df138a4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:18.813955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4cdb5f2d-e96e-4a78-97c6-3c500df138a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:18.901743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 106/212] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:18.902174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93bba8-abaf-47cc-9cb3-195d496970f8 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.902460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93bba8-abaf-47cc-9cb3-195d496970f8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.902736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93bba8-abaf-47cc-9cb3-195d496970f8 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.904401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 107/213] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:18.917530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:18.917817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:18.918094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:18.938841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:18.940165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:18.942108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.012612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 107/214] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:19.052951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93bba8-abaf-47cc-9cb3-195d496970f8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.139789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.165248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93bba8-abaf-47cc-9cb3-195d496970f8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.191943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.201936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.202901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.202901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.203760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.204794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.206271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.208017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 108/215] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.213467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.323480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f1823-31d1-4e37-be2c-4ca7a7015363 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.327470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5d087-55d9-4d38-9402-ea1808f16247 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.362423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.363122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.363434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.363706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.368868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.369751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.370121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.370594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.371167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.371576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.373277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.376299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:19.381414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:19.382332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:19.383120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 108/216] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:19.384804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.399561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.412135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 109/217] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:44:19.412135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.509622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3fe54-6fa7-4f52-a3d8-28a55a0302c8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.540424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5739e13-128b-478c-a201-59dcadf4cd40 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.549151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18daa59f-811b-425e-bd8b-c3e3f295471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.549846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daa59f-811b-425e-bd8b-c3e3f295471a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.550126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daa59f-811b-425e-bd8b-c3e3f295471a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.550394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18daa59f-811b-425e-bd8b-c3e3f295471a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.561874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18daa59f-811b-425e-bd8b-c3e3f295471a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-18daa59f-811b-425e-bd8b-c3e3f295471a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:19.567549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:19.568475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:19.569292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 109/218] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:19.569964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ff2c0-b0bf-47b3-a5e8-1af2ff56a99d None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.570533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ff2c0-b0bf-47b3-a5e8-1af2ff56a99d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.570533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ff2c0-b0bf-47b3-a5e8-1af2ff56a99d None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.572411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 110/219] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:19.577055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ff2c0-b0bf-47b3-a5e8-1af2ff56a99d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.589230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.590107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.590384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.590649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.591343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:19.591647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.593034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356924f8-af04-4a45-85de-c21606eddeab None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.602334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 110/220] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.603162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.603485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.603758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.624373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:19.626842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:19.652730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.702725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.712708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ff2c0-b0bf-47b3-a5e8-1af2ff56a99d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.715947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 111/221] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:19.716202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.755748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.756455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.756734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.757009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.757690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:19.758050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.760036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.770104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 111/222] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.775725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.825337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce063e66-3451-4375-af3f-18a85ca1d20f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.861161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.861720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.861999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.862272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] PATH_INFO: `/v3/users/6aa66c10b3344db0b1ea606935d8e58e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.879129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] RBAC: Authorizing `identity:get_user(user_id=6aa66c10b3344db0b1ea606935d8e58e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.880928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.882875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 112/223] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/users/6aa66c10b3344db0b1ea606935d8e58e => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.889217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.897736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3d206-80b2-46fd-a304-c1e0160b67fd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.929057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.955010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.962446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.963101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.963433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.963709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] PATH_INFO: `/v3/users/84d554f2aeb4485e9b94ca112d0934d6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.965040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] RBAC: Authorizing `identity:get_user(user_id=84d554f2aeb4485e9b94ca112d0934d6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.968178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.972117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 112/224] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/users/84d554f2aeb4485e9b94ca112d0934d6 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:19.980274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.046318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9e088-4dc7-4cff-ac5e-2fedd2e046ea None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.087190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.088303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.088588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.088861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.115671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1b38a-c4af-409b-bef0-8c67046fe4d2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.153591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.154249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.154527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.154796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.156055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.158153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.160093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 113/225] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.160912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.172645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.174108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.231836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.234012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.284783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3f89-a0e8-4224-96c4-be217e60af4b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.299656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 113/226] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:20.300478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.321133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.321770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.322096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.322373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.334516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:20.340168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:20.342352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 114/227] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:20.345327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.425869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149a318-768d-4880-af2d-5bc7f7545834 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.461219 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03a57d-0030-4441-8654-b14b8e7b985c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.461447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.462044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.462324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.462601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/84d554f2aeb4485e9b94ca112d0934d6/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.464577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef56326c836e4e1faf0cca5b864b5d30, user_id=84d554f2aeb4485e9b94ca112d0934d6, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.466030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.481469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.483766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 114/228] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:20 2022] PUT /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/84d554f2aeb4485e9b94ca112d0934d6/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:20.484843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.499806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.544656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.545310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.545588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.545865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.551544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.553255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.555626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 115/229] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.564359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.616723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a02e2-f261-4b63-9051-992c2cf34614 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.658715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.659796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.660292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.660743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.661819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:20.662398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.664587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.674883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 115/230] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.680832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.787168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.822831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315aa38-42f1-44c1-8552-88e16092fd5b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.832667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff99f40-79d6-45f1-8b60-830b8c0ee232 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.866816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.867461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.867750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.868330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.869348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=6aa66c10b3344db0b1ea606935d8e58e) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:20.869699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.871527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.873548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.874432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.875533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.876018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] PATH_INFO: `/v3/users/84d554f2aeb4485e9b94ca112d0934d6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.878259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] RBAC: Authorizing `identity:get_user(user_id=84d554f2aeb4485e9b94ca112d0934d6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.878617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 116/231] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/role_assignments?user.id=6aa66c10b3344db0b1ea606935d8e58e&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.880194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.882511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 116/232] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/users/84d554f2aeb4485e9b94ca112d0934d6 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.888326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.951476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:21.011560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e740e0-9e23-4dc3-88be-6df249d2559e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.049272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b2adb-1192-476b-ba97-df870e7f94f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.049927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b2adb-1192-476b-ba97-df870e7f94f3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.050250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b2adb-1192-476b-ba97-df870e7f94f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.050525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b2adb-1192-476b-ba97-df870e7f94f3 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.051764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b2adb-1192-476b-ba97-df870e7f94f3 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.053254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b2adb-1192-476b-ba97-df870e7f94f3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.055157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 117/233] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:21.060956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:21.191127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619bfbc-920a-4ad4-a60f-81bcad1ce44b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.230286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb50d0-f9b6-42b8-8886-17a5d9125003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.230894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb50d0-f9b6-42b8-8886-17a5d9125003 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.231176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb50d0-f9b6-42b8-8886-17a5d9125003 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.231485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb50d0-f9b6-42b8-8886-17a5d9125003 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/84d554f2aeb4485e9b94ca112d0934d6/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.233831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb50d0-f9b6-42b8-8886-17a5d9125003 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef56326c836e4e1faf0cca5b864b5d30, user_id=84d554f2aeb4485e9b94ca112d0934d6, role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.235871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb50d0-f9b6-42b8-8886-17a5d9125003 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.248309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:21.254325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 117/234] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:21 2022] PUT /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/84d554f2aeb4485e9b94ca112d0934d6/roles/e1b5b7a11157451ab80ef767124856f0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:22.347940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2ca29-01f4-47ff-bbc5-eb8ebcf04eb3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:22.348469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2ca29-01f4-47ff-bbc5-eb8ebcf04eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:22.348771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2ca29-01f4-47ff-bbc5-eb8ebcf04eb3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:22.350521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 118/235] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:22.356028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a6ad-8ce0-44c7-a033-532ab7f9bdc1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:22.356495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a6ad-8ce0-44c7-a033-532ab7f9bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:22.356577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75a6ad-8ce0-44c7-a033-532ab7f9bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:22.380233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3f75a6ad-8ce0-44c7-a033-532ab7f9bdc1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:22.381476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f75a6ad-8ce0-44c7-a033-532ab7f9bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:22.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 118/236] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:22.546472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1289c-3514-461c-bcf9-8c28e0b36a90 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:22.547366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1289c-3514-461c-bcf9-8c28e0b36a90 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:22.547651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1289c-3514-461c-bcf9-8c28e0b36a90 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:22.549391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 119/237] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:22.554718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469e5d3-eacb-4b70-a959-46c9fec74e94 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:22.555002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469e5d3-eacb-4b70-a959-46c9fec74e94 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:22.555304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469e5d3-eacb-4b70-a959-46c9fec74e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:22.580204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9469e5d3-eacb-4b70-a959-46c9fec74e94 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:22.581756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9469e5d3-eacb-4b70-a959-46c9fec74e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:22.679750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 119/238] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:22.855664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1289c-3514-461c-bcf9-8c28e0b36a90 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:22.996518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1289c-3514-461c-bcf9-8c28e0b36a90 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.017315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.039248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.039894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.040168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.040437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.057786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:23.060735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ada17-620c-452c-b8c0-f3cbd25ce8cf None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.061348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ada17-620c-452c-b8c0-f3cbd25ce8cf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.061825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ada17-620c-452c-b8c0-f3cbd25ce8cf None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:23.062641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:23.063553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:23.064336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 120/239] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:23.066383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 120/240] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:23.070694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.072201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.072697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.073187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.108883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:23.112306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:23.182007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.194873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 121/241] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:23.195863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.203852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fe58a-03d0-4fda-a7f0-babc0806d740 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.262583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.263118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.263422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.263694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.264363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:23.264666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.266585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.290210 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 121/242] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:23.295763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.338514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac61694-91ac-4ffb-bdd6-ecf275a3952d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.376886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.409346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.410006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.410260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.410531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.412356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.413825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.415411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 122/243] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:23.422110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.495628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58324d84-a1c3-4c62-901a-7147aa7988e7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.511012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.549917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.550549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.550829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.551104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] PATH_INFO: `/v3/users/267cab2fa52b47879446978b19474688` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.556605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a41358-b585-45c6-b5fb-579bec4e079a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.566881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] RBAC: Authorizing `identity:get_user(user_id=267cab2fa52b47879446978b19474688)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.569017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.571612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 122/244] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/users/267cab2fa52b47879446978b19474688 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:23.572799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.600626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.601311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.601596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.601930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.602605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.604785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.635758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 123/245] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:23 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:23.637435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.738588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.746160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422dbf-9a43-40d0-aa67-bb825184f427 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.785898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacabf7b-e3ed-4ce6-9474-3beb02513326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.786916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabf7b-e3ed-4ce6-9474-3beb02513326 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.787444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabf7b-e3ed-4ce6-9474-3beb02513326 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.787724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacabf7b-e3ed-4ce6-9474-3beb02513326 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.793538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d8e86-7af9-4f20-8ef5-be2d4f1de488 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.810655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacabf7b-e3ed-4ce6-9474-3beb02513326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:23.814476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-eacabf7b-e3ed-4ce6-9474-3beb02513326 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:23.816796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:23.817922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:23.819094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 123/246] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:23.824130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cfe11-fbea-42c3-be30-12ffb1758a64 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.824428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cfe11-fbea-42c3-be30-12ffb1758a64 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.824714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cfe11-fbea-42c3-be30-12ffb1758a64 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.826413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 124/247] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:23.828090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cfe11-fbea-42c3-be30-12ffb1758a64 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.859021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.860773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.861154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.861650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.946518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.948334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.972074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cfe11-fbea-42c3-be30-12ffb1758a64 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.017668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.018556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.019010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.019599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.020522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:24.020833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.022314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1988f-4046-4d9c-9ffb-c7d0c6655da7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.034180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 125/248] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.039022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.039022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.039022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.057303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:24.058522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:24.080964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.094183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 124/249] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:24.095410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.114987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.162324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 126/250] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:24.165115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.234594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c7681-3ed5-4ba6-b995-ba840b8129a4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.272929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc7b4c-e893-446f-a307-d9ca921c1d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.273676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc7b4c-e893-446f-a307-d9ca921c1d4e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.273939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc7b4c-e893-446f-a307-d9ca921c1d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.274208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc7b4c-e893-446f-a307-d9ca921c1d4e None admin] PATH_INFO: `/v3/users/84d554f2aeb4485e9b94ca112d0934d6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.275490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc7b4c-e893-446f-a307-d9ca921c1d4e None admin] RBAC: Authorizing `identity:get_user(user_id=84d554f2aeb4485e9b94ca112d0934d6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.276934 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc7b4c-e893-446f-a307-d9ca921c1d4e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.278860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 125/251] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/users/84d554f2aeb4485e9b94ca112d0934d6 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.280553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37aa6d-2f9f-464d-8a38-ccde6079bec9 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.281686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37aa6d-2f9f-464d-8a38-ccde6079bec9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.282281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37aa6d-2f9f-464d-8a38-ccde6079bec9 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.287459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 126/252] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:24.291324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37aa6d-2f9f-464d-8a38-ccde6079bec9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.297251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce409fc4-5d84-42ce-b81a-0ff51b6d7344 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.349272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3201-ebaa-4d13-8801-8f2c9a8efc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.349857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3201-ebaa-4d13-8801-8f2c9a8efc0e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.350128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3201-ebaa-4d13-8801-8f2c9a8efc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.350394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3201-ebaa-4d13-8801-8f2c9a8efc0e None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.351549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3201-ebaa-4d13-8801-8f2c9a8efc0e None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.353556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3201-ebaa-4d13-8801-8f2c9a8efc0e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.355600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 127/253] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.356791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.357376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.357643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.389886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:24.394779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:24.418149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.439232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.476317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37aa6d-2f9f-464d-8a38-ccde6079bec9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.501196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 128/254] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:24.502476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.534448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.535276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.535557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.535824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.663329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.664720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.698112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c4e14-9dc8-4763-8f8d-9c48c6931d2e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.760032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.760712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.760997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.761270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.762010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] RBAC: Adding query filter params (role.id=d92a531856fd46cc87fffc7a3af54ac7, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=267cab2fa52b47879446978b19474688) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:24.762316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.763801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.777030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 129/255] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/role_assignments?user.id=267cab2fa52b47879446978b19474688&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=d92a531856fd46cc87fffc7a3af54ac7 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.779055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.805348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.826492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.853051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 127/256] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:24.854533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.914706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37cbb1-457e-42cb-bb4d-723fbb52d64b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.952015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.952734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.953065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.953385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.965466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:24.971108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:24.972076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:24.972842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 130/257] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:24.973497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.982923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495fb28-0502-45aa-a680-7cd449cae1ec None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.026253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.027310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.027795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.028292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.040812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:25.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:25.049313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:25.052017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 128/258] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:25.052673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.061659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.104307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0e1c0-5754-4d1f-92cd-0573263fc5e1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.125281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.146782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.147796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.148713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.149182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.150672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.152367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.154577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 131/259] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.155794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.266348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8b5fe-a450-410a-85a2-a50b3ec7c539 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.270702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb5d02-a0d7-4363-b226-a633c199ecbc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.306883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.307876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.308388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.308865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.309730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:25.310246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.311934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.317925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.322778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 132/260] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.323992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.339921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.340550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.340835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.341106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.341786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:25.342292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.344026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.369425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 129/261] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.370237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.450471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1c2d7-216a-45da-9fae-5e6012dad150 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.525988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.527072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.527620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.528082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.529015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=84d554f2aeb4485e9b94ca112d0934d6) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:25.529612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.531913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.542131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb79d7-55d7-4ada-8dde-add8307a2912 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.544445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.547612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 133/262] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/role_assignments?user.id=84d554f2aeb4485e9b94ca112d0934d6&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.548692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.582673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.583181 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.583498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.583773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] PATH_INFO: `/v3/users/84d554f2aeb4485e9b94ca112d0934d6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.585050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] RBAC: Authorizing `identity:get_user(user_id=84d554f2aeb4485e9b94ca112d0934d6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.586602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.591231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 130/263] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/users/84d554f2aeb4485e9b94ca112d0934d6 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.596422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.693870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.727768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce55edc-cc4e-4a00-9b42-48151e5ae572 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.747549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e6cf8-7ec8-4d5b-b8b8-be9b695093ad None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.788365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.788567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.788916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.789196 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.789470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.789568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.789978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.790560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] PATH_INFO: `/v3/users/6aa66c10b3344db0b1ea606935d8e58e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.792665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] RBAC: Authorizing `identity:get_user(user_id=6aa66c10b3344db0b1ea606935d8e58e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.794304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.796055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 134/264] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/users/6aa66c10b3344db0b1ea606935d8e58e => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.801707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.881570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.914236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8338-6546-475e-915a-09d9db90a921 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.951988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.953717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.954007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.954289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.955582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.957329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.959633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 135/265] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.960141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.966782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.968038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.969622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.077173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93efc82-810f-4fac-a281-d12e67485bb9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.116038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.116703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.116984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.117257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/6aa66c10b3344db0b1ea606935d8e58e/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.119317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef56326c836e4e1faf0cca5b864b5d30, user_id=6aa66c10b3344db0b1ea606935d8e58e, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.120786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.143617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 136/266] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:25 2022] PUT /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/6aa66c10b3344db0b1ea606935d8e58e/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:26.186165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 131/267] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:26.192801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.263662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.310540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fb7dd-7e86-49cd-b673-d40ad5355fc9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.348715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d49e74-6bf4-4728-8ec5-6ee24ad65558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.349283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d49e74-6bf4-4728-8ec5-6ee24ad65558 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.349562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d49e74-6bf4-4728-8ec5-6ee24ad65558 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.349835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d49e74-6bf4-4728-8ec5-6ee24ad65558 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.351035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d49e74-6bf4-4728-8ec5-6ee24ad65558 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.352555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d49e74-6bf4-4728-8ec5-6ee24ad65558 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.359648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 137/268] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.361034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.439742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.472589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e881a0-0009-4210-abca-cfa912ca8144 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.511061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f60352-9666-4511-9787-d627c69685fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.511711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60352-9666-4511-9787-d627c69685fb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.511991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60352-9666-4511-9787-d627c69685fb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.512267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60352-9666-4511-9787-d627c69685fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.513006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f60352-9666-4511-9787-d627c69685fb None admin] RBAC: Adding query filter params (role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=84d554f2aeb4485e9b94ca112d0934d6) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:26.513316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f60352-9666-4511-9787-d627c69685fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.514772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f60352-9666-4511-9787-d627c69685fb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.524540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 132/269] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/role_assignments?user.id=84d554f2aeb4485e9b94ca112d0934d6&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.838438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab9308-501f-4989-aaf3-0670e22100d9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.838960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab9308-501f-4989-aaf3-0670e22100d9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.838960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab9308-501f-4989-aaf3-0670e22100d9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.840708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 138/270] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:26.846166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.846662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.846739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.866657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:26.867920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:26.882908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c7a7f-532e-4bfd-b3b5-73960ff8a27d None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.883258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c7a7f-532e-4bfd-b3b5-73960ff8a27d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.883494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c7a7f-532e-4bfd-b3b5-73960ff8a27d None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.885101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 139/271] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:26.897559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.897998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.898437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.918136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:26.919580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:26.964898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 133/272] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:27.003145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 140/273] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:27.243594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.293036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.326364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.363960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb6c9-77fc-4992-a7bd-60e3b18c71ad None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.380644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.401529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f360ecc-59f5-4e7b-81ad-f01481334efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.402160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f360ecc-59f5-4e7b-81ad-f01481334efc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.402435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f360ecc-59f5-4e7b-81ad-f01481334efc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.402705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f360ecc-59f5-4e7b-81ad-f01481334efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.413984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f360ecc-59f5-4e7b-81ad-f01481334efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7f360ecc-59f5-4e7b-81ad-f01481334efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:27.418825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:27.419796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:27.420578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 134/274] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:27.421311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eed07-a1d1-438b-96f7-9e6db18023d2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.421311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eed07-a1d1-438b-96f7-9e6db18023d2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.423307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eed07-a1d1-438b-96f7-9e6db18023d2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.427448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 135/275] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:27.428209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eed07-a1d1-438b-96f7-9e6db18023d2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.441952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5c428-59c7-47dc-b036-892ff612964e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.488022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac443b-4e53-4a01-aade-7086e02be642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.488886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac443b-4e53-4a01-aade-7086e02be642 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.489313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac443b-4e53-4a01-aade-7086e02be642 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.489717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac443b-4e53-4a01-aade-7086e02be642 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.491027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac443b-4e53-4a01-aade-7086e02be642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.492692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac443b-4e53-4a01-aade-7086e02be642 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.494434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 141/276] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.495720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.496162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.496588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.517687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:27.519274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:27.548857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eed07-a1d1-438b-96f7-9e6db18023d2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.579111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.588077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 142/277] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:27.589076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.596312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.596999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.597345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.597680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.598421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:27.598786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.600339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.615727 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 136/278] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.617067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.661505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.708114 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346c84f-b43c-4c20-8bab-6c14f0c81933 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.744215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.744984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.745203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.745464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.746161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.747710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.763982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97173b2-7592-4c07-83ee-eb885edceffe None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.774498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 143/279] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:27 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:27.806506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.807456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.807912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.808335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] PATH_INFO: `/v3/users/f5dce76c9fd8469ea65892d3b6e9ef66` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.824703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] RBAC: Authorizing `identity:get_user(user_id=f5dce76c9fd8469ea65892d3b6e9ef66)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.826443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.828944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 137/280] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/users/f5dce76c9fd8469ea65892d3b6e9ef66 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.835145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.888874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.949968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.983843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6e688-432b-42f3-8075-74c6ecb38af3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.001647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.033645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d643e91-89dd-4ea8-9c20-d67c48828f8f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.040860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.041538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.041818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.042085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.043332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, user.id=f5dce76c9fd8469ea65892d3b6e9ef66, scope.system=all) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:28.043645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.045662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.053540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 144/281] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/role_assignments?user.id=f5dce76c9fd8469ea65892d3b6e9ef66&scope.system=all&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:28.084439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.085178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.085454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.085719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.098750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:28.104410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:28.105367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:28.106147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 138/282] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:28.112990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.243746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6cb45-47bf-4ca1-8105-992d59935ad3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.296146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.297447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.297736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.298065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.298967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:28.299322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.300726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.310343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 145/283] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:28.316510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.467766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e8d7-12e7-470d-a540-cb6e3b885f1b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.478715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.521356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.522400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.522873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.523366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] PATH_INFO: `/v3/users/267cab2fa52b47879446978b19474688` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.524689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] RBAC: Authorizing `identity:get_user(user_id=267cab2fa52b47879446978b19474688)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.526405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.528514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 139/284] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/users/267cab2fa52b47879446978b19474688 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:28.534461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.605038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.679290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca3d13-7e4f-4137-8d00-33dc73247ce9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.725831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13115f-7219-424f-b36f-473b40a17dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.726486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13115f-7219-424f-b36f-473b40a17dec None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.726771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13115f-7219-424f-b36f-473b40a17dec None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.727048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13115f-7219-424f-b36f-473b40a17dec None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.728343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13115f-7219-424f-b36f-473b40a17dec None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.729848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13115f-7219-424f-b36f-473b40a17dec None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.736180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 146/285] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:28.742304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.892918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce936d8-b911-43bc-b2bb-7206a0105877 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.939436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165027b6-c039-47f6-a51a-3bc3ef6637e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.940178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165027b6-c039-47f6-a51a-3bc3ef6637e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.940456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165027b6-c039-47f6-a51a-3bc3ef6637e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.940735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165027b6-c039-47f6-a51a-3bc3ef6637e6 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/267cab2fa52b47879446978b19474688/roles/d92a531856fd46cc87fffc7a3af54ac7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.948742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165027b6-c039-47f6-a51a-3bc3ef6637e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef56326c836e4e1faf0cca5b864b5d30, user_id=267cab2fa52b47879446978b19474688, role_id=d92a531856fd46cc87fffc7a3af54ac7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.950233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165027b6-c039-47f6-a51a-3bc3ef6637e6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.969305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 140/286] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:28 2022] PUT /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/users/267cab2fa52b47879446978b19474688/roles/d92a531856fd46cc87fffc7a3af54ac7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:28.992632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a538a8-2a19-40fc-9b11-5c76d94fc1ee None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.993175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a538a8-2a19-40fc-9b11-5c76d94fc1ee None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.993175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a538a8-2a19-40fc-9b11-5c76d94fc1ee None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.995121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 147/287] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:29.001171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.001431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.001710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.028453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:29.029697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:29.068238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:29.137104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 141/288] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:29.466602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a538a8-2a19-40fc-9b11-5c76d94fc1ee None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:29.582772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a538a8-2a19-40fc-9b11-5c76d94fc1ee None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:29.590772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:29.630733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:29.631404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.631688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.632032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.645665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:29.650657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:29.651554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:29.652407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 148/289] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:29.658203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:29.772260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad8a5f-fb47-4da3-95ba-f65459f15112 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:29.814713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:29.815353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.815632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.815904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.816566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:29.816866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:29.818234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:29.831072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 142/290] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:29.832369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:30.006918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff69dc8-1c6a-4ab4-a1bc-aca191028172 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:30.059843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:30.060483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.060758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.061030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] PATH_INFO: `/v3/users/6aa66c10b3344db0b1ea606935d8e58e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.062213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] RBAC: Authorizing `identity:get_user(user_id=6aa66c10b3344db0b1ea606935d8e58e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:30.063708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:30.065603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 149/291] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:29 2022] GET /identity/v3/users/6aa66c10b3344db0b1ea606935d8e58e => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:30.071145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:30.108660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:30.198294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdb164-7b7c-49d7-abf2-a617db8525d6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:30.234089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:30.234708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.234985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.235282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.332013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:30.333432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:30.367996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:30.465309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 143/292] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:30.472302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:30.547337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:30.644844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf7638-84f9-4900-a572-993dab86969f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:30.688844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe74672-8f4f-4be5-8f1b-cc1ebf6b4ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:30.689489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe74672-8f4f-4be5-8f1b-cc1ebf6b4ddc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.689770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe74672-8f4f-4be5-8f1b-cc1ebf6b4ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.690045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe74672-8f4f-4be5-8f1b-cc1ebf6b4ddc None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.691176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe74672-8f4f-4be5-8f1b-cc1ebf6b4ddc None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:30.692660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe74672-8f4f-4be5-8f1b-cc1ebf6b4ddc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:30.694686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 150/293] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:30 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:30.700550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:30.725253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec5f2-5b1f-4948-a2f0-3e6ee4008bb4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.725536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec5f2-5b1f-4948-a2f0-3e6ee4008bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.725806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ec5f2-5b1f-4948-a2f0-3e6ee4008bb4 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.727152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 151/294] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:30.733450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ab30-d2c6-431d-b9cb-dcce4e87b412 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.733730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ab30-d2c6-431d-b9cb-dcce4e87b412 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.734519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ab30-d2c6-431d-b9cb-dcce4e87b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.758323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7bb2ab30-d2c6-431d-b9cb-dcce4e87b412 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:30.759811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7bb2ab30-d2c6-431d-b9cb-dcce4e87b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:30.828787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 152/295] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:30.830259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57ba83-7ae0-4298-8739-f19aa8f0ad74 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.830651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57ba83-7ae0-4298-8739-f19aa8f0ad74 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.830945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57ba83-7ae0-4298-8739-f19aa8f0ad74 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 153/296] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad401056-4c69-48be-8b17-7ade801fdbfb None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad401056-4c69-48be-8b17-7ade801fdbfb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad401056-4c69-48be-8b17-7ade801fdbfb None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 154/297] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.839293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.839793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.847247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063a32-85f5-4164-ae60-bf779c9957d5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:30.862362 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:30.863878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:30.888888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:30.902513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:30.906254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:30.906871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.907151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.907452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.908231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=6aa66c10b3344db0b1ea606935d8e58e) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:30.908498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:30.909959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bc852-7de4-4926-96fa-e9deb85c63f9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:30.923232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 144/298] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:30 2022] GET /identity/v3/role_assignments?user.id=6aa66c10b3344db0b1ea606935d8e58e&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:30.923232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.923232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.923232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.951212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:30.952459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:30.976894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 155/299] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:31.058041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 145/300] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:31.101021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.212784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18186-27aa-4ec1-90b1-90ebcf79b1a6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.248159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.248803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.249078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.249354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.260631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:31.266135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:31.267010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:31.267828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 156/301] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:31.273273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.357332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.393065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.394186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac122d-5946-4a70-b1f2-46370dd472ee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.419659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.442941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.443447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.443730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.444001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.444852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:31.445530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.447228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.457751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 146/302] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.458438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.501861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783b1d-af8c-4020-a168-d7332c485206 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.541582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.542217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.542496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.542768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.556955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:31.561893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:31.563433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:31.564204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 157/303] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:31.564945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.573483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcc5c3-a6e9-4e52-b2b7-3b91a6e83855 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.612469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.613105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.613442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.613664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.615228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.617021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.619009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 147/304] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.619749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.668654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.681330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.689464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029c02e-3be0-453b-88bc-a2f78c969194 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.727864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.728514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.728787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.729053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] PATH_INFO: `/v3/users/f5dce76c9fd8469ea65892d3b6e9ef66` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.730451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] RBAC: Authorizing `identity:get_user(user_id=f5dce76c9fd8469ea65892d3b6e9ef66)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.732064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.734004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 158/305] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/users/f5dce76c9fd8469ea65892d3b6e9ef66 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.734951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.742290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ded04-0fe1-468e-bf09-5ab5c0c903cc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.782959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.783620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.783900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.784438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.784853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:31.785160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.786606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.797169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 148/306] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.797830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.855331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b57be-d653-4f6a-9387-0ebdcc627611 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.894628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.895269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.895552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.895823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] PATH_INFO: `/v3/groups` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.896460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.897920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.913390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3968c-0648-4e95-80b7-ee106b731a70 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.915232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 159/307] 10.222.0.66 () {60 vars in 1134 bytes} [Thu Sep 22 14:44:31 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:31.915570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.929536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.930640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.957987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f4fb0-6bec-4419-abab-5728e6bf4eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.958683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f4fb0-6bec-4419-abab-5728e6bf4eda None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.958963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f4fb0-6bec-4419-abab-5728e6bf4eda None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.959268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f4fb0-6bec-4419-abab-5728e6bf4eda None admin] PATH_INFO: `/v3/system/users/f5dce76c9fd8469ea65892d3b6e9ef66/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.960739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f4fb0-6bec-4419-abab-5728e6bf4eda None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f5dce76c9fd8469ea65892d3b6e9ef66, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.962295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f4fb0-6bec-4419-abab-5728e6bf4eda None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.973444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 149/308] 10.222.0.66 () {58 vars in 1334 bytes} [Thu Sep 22 14:44:31 2022] PUT /identity/v3/system/users/f5dce76c9fd8469ea65892d3b6e9ef66/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:31.974646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76e9e-4c7d-4af3-b734-9c031aae3421 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.974935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76e9e-4c7d-4af3-b734-9c031aae3421 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.975294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76e9e-4c7d-4af3-b734-9c031aae3421 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.982846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 150/309] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:31.985627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.985918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.986195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.007746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:32.009345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:32.064104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaa28-f709-4be8-b76f-cb466759609e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.111640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.112343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.112568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.112824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] PATH_INFO: `/v3/users/8c3e040fa002409cbfe00506b1061c8e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.125005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 151/310] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:32.130717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=8c3e040fa002409cbfe00506b1061c8e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.132246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.134595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 160/311] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/users/8c3e040fa002409cbfe00506b1061c8e => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.141477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.247239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62178b1-eb62-4322-9b2c-a3813d3d6626 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.255249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.287469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.288118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.288338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.288598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.289320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] RBAC: Adding query filter params (role.id=d92a531856fd46cc87fffc7a3af54ac7, user.id=8c3e040fa002409cbfe00506b1061c8e, scope.system=all) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:32.289620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.291064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.297027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 152/312] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/role_assignments?user.id=8c3e040fa002409cbfe00506b1061c8e&scope.system=all&role.id=d92a531856fd46cc87fffc7a3af54ac7 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.413642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.492406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.528677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6360d7-e2a7-43dd-82e8-88fe0f42bbb4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.566214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.566850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.567124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.567425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.578465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:32.583532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:32.584391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:32.585306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 161/313] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:32.590702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.706994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad668a-eb25-4502-a224-20e4b68e2da5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.743218 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.744130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.744584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.745027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.745882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:32.746356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.747925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.758047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 153/314] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.764265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.869472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3359d-9dd8-42c7-a72d-459b0685bd2b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.900384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.908672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.909301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.909575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.909845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] PATH_INFO: `/v3/users/267cab2fa52b47879446978b19474688` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.910972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] RBAC: Authorizing `identity:get_user(user_id=267cab2fa52b47879446978b19474688)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.912506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.914092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 162/315] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/users/267cab2fa52b47879446978b19474688 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.919651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.960619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:33.041380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96359796-0947-4ed6-ab50-dcb02dcd30be None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.079485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.080467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.080952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.081419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.163101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.164907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.197599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:33.276296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 154/316] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:33.283249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:33.397069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97fd5f6-7b33-4eee-bdf7-fee2141483a4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.432963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c4eaf-9a5b-4c7f-9fa0-091d664bd7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.433579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c4eaf-9a5b-4c7f-9fa0-091d664bd7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.433864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c4eaf-9a5b-4c7f-9fa0-091d664bd7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.434136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c4eaf-9a5b-4c7f-9fa0-091d664bd7a0 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.435290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c4eaf-9a5b-4c7f-9fa0-091d664bd7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.436949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c4eaf-9a5b-4c7f-9fa0-091d664bd7a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.438597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 163/317] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:33.444525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:33.569407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699a152-9814-499e-b50e-f452c1469391 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.601251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:33.606975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.607614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.608057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.608338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.609625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] RBAC: Adding query filter params (role.id=d92a531856fd46cc87fffc7a3af54ac7, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30, user.id=267cab2fa52b47879446978b19474688) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:33.609941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.611470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dabf2-387b-4b84-b021-08130a111722 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.618118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 155/318] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/role_assignments?user.id=267cab2fa52b47879446978b19474688&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=d92a531856fd46cc87fffc7a3af54ac7 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:34.256555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f1067-3963-4a2f-8df3-84b0bd4ff63a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.256945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f1067-3963-4a2f-8df3-84b0bd4ff63a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.257146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f1067-3963-4a2f-8df3-84b0bd4ff63a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.258434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 164/319] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:34.263926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114b44-3f76-43b6-b407-fd74b46d87c4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.264215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114b44-3f76-43b6-b407-fd74b46d87c4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.264488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114b44-3f76-43b6-b407-fd74b46d87c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.283334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-17114b44-3f76-43b6-b407-fd74b46d87c4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:34.284507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-17114b44-3f76-43b6-b407-fd74b46d87c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:34.367259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 156/320] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:34.426204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90639f-f365-45a3-a724-ff46554e27ce None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.426737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90639f-f365-45a3-a724-ff46554e27ce None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.426737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90639f-f365-45a3-a724-ff46554e27ce None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.428246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 165/321] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:34.433395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556e3bd-f0dc-4252-92c5-a7e8a8fbb199 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.433723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556e3bd-f0dc-4252-92c5-a7e8a8fbb199 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.433995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556e3bd-f0dc-4252-92c5-a7e8a8fbb199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.453323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0556e3bd-f0dc-4252-92c5-a7e8a8fbb199 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:34.454552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0556e3bd-f0dc-4252-92c5-a7e8a8fbb199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:34.537414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 157/322] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:34.605902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879294a-fc7d-4dd1-bb22-48c5419832ac None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.606208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879294a-fc7d-4dd1-bb22-48c5419832ac None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.606511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879294a-fc7d-4dd1-bb22-48c5419832ac None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.608059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 166/323] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:34.613563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.613858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.614135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.631929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879294a-fc7d-4dd1-bb22-48c5419832ac None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.633312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:34.634846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:34.691810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:34.698400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:34.719256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 158/324] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:34.744189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879294a-fc7d-4dd1-bb22-48c5419832ac None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:34.779895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:34.780609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.780821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.781088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.792429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:34.797771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:34.799050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:34.799912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:34.803415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 167/325] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:34.804693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.806050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.913534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9318-fbde-4a2b-9810-3943160150dd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:34.920104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa3b1b-9137-4386-a581-43dd83777a1c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:34.938363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:34.950589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:34.951235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.951525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.951804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.952947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:34.954409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:34.956012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 168/326] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:34.960078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:34.960764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.961039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.961305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.961468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.961958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:34.962255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:34.963689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:34.973809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 159/327] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:34.979361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.028012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.066682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b27ff-b42d-476c-9080-794e369f8c7e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.079534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81e968-a7b0-49f0-ac1a-45216e0d8838 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.100805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.101414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.101692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.101963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] PATH_INFO: `/v3/groups` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.102588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.104167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.115071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.115740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.116022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.116299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] PATH_INFO: `/v3/users/f5dce76c9fd8469ea65892d3b6e9ef66` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.117433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] RBAC: Authorizing `identity:get_user(user_id=f5dce76c9fd8469ea65892d3b6e9ef66)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.118867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.119574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 169/328] 10.222.0.66 () {60 vars in 1134 bytes} [Thu Sep 22 14:44:34 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:44:35.120391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.121073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 160/329] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:34 2022] GET /identity/v3/users/f5dce76c9fd8469ea65892d3b6e9ef66 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:35.137061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.184467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.245869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df709e2-9bdf-4a03-8a25-d7bd89e5b570 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.259763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55830-2d4e-4a79-bcd1-3d96919b67ca None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.278658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.285761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.286704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.287137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.287587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.298761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:35.299971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.300590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.301694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.301694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.301936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] RBAC: Adding query filter params (role.id=074c5bafaca74f698537cbd8c86e21e3, user.id=f5dce76c9fd8469ea65892d3b6e9ef66, scope.system=all) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:35.302228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.303708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:35.304784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:35.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:35.306438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 170/330] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:35.311991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.320687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 161/331] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/role_assignments?user.id=f5dce76c9fd8469ea65892d3b6e9ef66&scope.system=all&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:35.439598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14316d6b-cb5d-48e6-9547-9f603e11c4aa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.456157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.490465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.491100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.491420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.491751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.492448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:35.492751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.494211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.505899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 171/332] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:35.511697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.629062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f67dd1-77c8-4920-9a22-07b71a88c066 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.669478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d1ef-acda-42b1-9dae-afad9ef2eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.670115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d1ef-acda-42b1-9dae-afad9ef2eb9b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.670395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d1ef-acda-42b1-9dae-afad9ef2eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.670669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d1ef-acda-42b1-9dae-afad9ef2eb9b None admin] PATH_INFO: `/v3/users/8c3e040fa002409cbfe00506b1061c8e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.673318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7d1ef-acda-42b1-9dae-afad9ef2eb9b None admin] RBAC: Authorizing `identity:get_user(user_id=8c3e040fa002409cbfe00506b1061c8e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.674774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7d1ef-acda-42b1-9dae-afad9ef2eb9b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.676710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 162/333] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/users/8c3e040fa002409cbfe00506b1061c8e => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:35.682383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.791653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c412969-4739-4ec3-9eb1-590506fde87d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.828901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465fa4d-ff11-49e3-8597-5455b0448094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.829562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465fa4d-ff11-49e3-8597-5455b0448094 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.829844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465fa4d-ff11-49e3-8597-5455b0448094 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.830116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465fa4d-ff11-49e3-8597-5455b0448094 None admin] PATH_INFO: `/v3/system/users/8c3e040fa002409cbfe00506b1061c8e/roles/d92a531856fd46cc87fffc7a3af54ac7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.831600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465fa4d-ff11-49e3-8597-5455b0448094 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8c3e040fa002409cbfe00506b1061c8e, role_id=d92a531856fd46cc87fffc7a3af54ac7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.833007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465fa4d-ff11-49e3-8597-5455b0448094 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.843099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 172/334] 10.222.0.66 () {58 vars in 1334 bytes} [Thu Sep 22 14:44:35 2022] PUT /identity/v3/system/users/8c3e040fa002409cbfe00506b1061c8e/roles/d92a531856fd46cc87fffc7a3af54ac7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:37.616982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb10d2-b963-4883-b0f2-ae346dea8729 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:37.617967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb10d2-b963-4883-b0f2-ae346dea8729 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:37.618501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb10d2-b963-4883-b0f2-ae346dea8729 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:37.620444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 163/335] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:37.625871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:37.626747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:37.627239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:37.652273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:37.653816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:37.690398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:37.744710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 173/336] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:38.257793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb10d2-b963-4883-b0f2-ae346dea8729 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:38.316689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:38.391972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb10d2-b963-4883-b0f2-ae346dea8729 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:38.433280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8350-592b-4a5b-9d0d-96cdf74cf2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:38.433908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8350-592b-4a5b-9d0d-96cdf74cf2cb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.434182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8350-592b-4a5b-9d0d-96cdf74cf2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.434446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8350-592b-4a5b-9d0d-96cdf74cf2cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.448629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d8350-592b-4a5b-9d0d-96cdf74cf2cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ca4d8350-592b-4a5b-9d0d-96cdf74cf2cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:38.454069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:38.454983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:38.455826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 164/337] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:38.462056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:38.635754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b3a6b-fc4d-407e-a761-a288bca02784 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:38.639348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a954ca8-ab23-4383-9007-7b6b978d6d5a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.639872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a954ca8-ab23-4383-9007-7b6b978d6d5a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.640024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a954ca8-ab23-4383-9007-7b6b978d6d5a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.641265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 165/338] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:38.646466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb207-6a76-483c-a3b1-1c6e49132fb0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.646754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb207-6a76-483c-a3b1-1c6e49132fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.647026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbb207-6a76-483c-a3b1-1c6e49132fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.682821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:38.683576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.683858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.684127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.684796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:38.685094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:38.686487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:38.700262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3cdbb207-6a76-483c-a3b1-1c6e49132fb0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:38.700576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 174/339] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:38.701463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3cdbb207-6a76-483c-a3b1-1c6e49132fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:38.706191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:38.750340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:38.861979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 166/340] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:38.868986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bd0b1-4639-49ce-9af2-bc0fa55b5778 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:38.929388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0627cf1-a6a3-4f64-bf4f-5379942f00eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:38.930417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0627cf1-a6a3-4f64-bf4f-5379942f00eb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.930897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0627cf1-a6a3-4f64-bf4f-5379942f00eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.931418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0627cf1-a6a3-4f64-bf4f-5379942f00eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.020809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0627cf1-a6a3-4f64-bf4f-5379942f00eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.022620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0627cf1-a6a3-4f64-bf4f-5379942f00eb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.055067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.096629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e52b-4316-4736-9fd5-4b6d34aaab32 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.096949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e52b-4316-4736-9fd5-4b6d34aaab32 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.097184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e52b-4316-4736-9fd5-4b6d34aaab32 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.098539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 167/341] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:39.104028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6fdb5-bcfb-4948-998b-5163d1069e81 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.104202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6fdb5-bcfb-4948-998b-5163d1069e81 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.104466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6fdb5-bcfb-4948-998b-5163d1069e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.131162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cce6fdb5-bcfb-4948-998b-5163d1069e81 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:39.132416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cce6fdb5-bcfb-4948-998b-5163d1069e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:39.145591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 175/342] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:39.147032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0351da-928d-44b6-97b7-fe3f63294e9d None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.147358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0351da-928d-44b6-97b7-fe3f63294e9d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.147632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0351da-928d-44b6-97b7-fe3f63294e9d None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.148987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 176/343] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:39.156743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0351da-928d-44b6-97b7-fe3f63294e9d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.181375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.258383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 168/344] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:39.259982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.260270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.260548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.294233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:39.295483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:39.366783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0351da-928d-44b6-97b7-fe3f63294e9d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.414464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.417322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 169/345] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:39.418199 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.430740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.431521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.431743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.432016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.434838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.436322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.444203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 177/346] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:39.447703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.613238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.619648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a85fa-d3ec-4c6d-a9f4-45ee64f59a19 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.630766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35831192-f3a2-4999-b482-cb7b5289d068 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.675741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4275aaf-952b-45f6-8c62-aeaba5c29cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.677064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.678208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4275aaf-952b-45f6-8c62-aeaba5c29cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.679039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.679540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4275aaf-952b-45f6-8c62-aeaba5c29cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.680061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4275aaf-952b-45f6-8c62-aeaba5c29cd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.680536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.681019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.697067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.698626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.701356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 178/347] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:39.702239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.708229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4275aaf-952b-45f6-8c62-aeaba5c29cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a4275aaf-952b-45f6-8c62-aeaba5c29cd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:39.713292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:39.714326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:39.715185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 170/348] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:39.718737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae467570-8dd3-4625-8c06-603055aecbd3 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.718737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae467570-8dd3-4625-8c06-603055aecbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.718737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae467570-8dd3-4625-8c06-603055aecbd3 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.721171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 171/349] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:39.722418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae467570-8dd3-4625-8c06-603055aecbd3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.806711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.856872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d1237-844f-48ea-bb12-2b436eddcd2d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.866006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae467570-8dd3-4625-8c06-603055aecbd3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.915058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b8f2-ffc2-4733-b00b-e1840393405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.916016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b8f2-ffc2-4733-b00b-e1840393405d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.916016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b8f2-ffc2-4733-b00b-e1840393405d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.916306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b8f2-ffc2-4733-b00b-e1840393405d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.924758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.925631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.926000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.926321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.927184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:39.927575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.930656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.934522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b8f2-ffc2-4733-b00b-e1840393405d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5e77b8f2-ffc2-4733-b00b-e1840393405d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:39.940912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:39.941901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:39.942778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 179/350] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:39.943982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.944087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.944286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.945083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 172/351] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:39.946182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.963717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:39.965059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:39.968938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.082098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 180/352] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:40.083329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.099189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a13eb-bfc4-4852-afa1-be747eeeb817 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.185258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.185751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.186129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.187494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.187983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.188728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:40.189127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.190567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.209019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 173/353] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:40.209791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.264417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af398f-bbde-45ef-b632-11caeb911972 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.316681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.318122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.318615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.318959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.320010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.336564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:40.343447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:40.344356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:40.345142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:40.345746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e065b6-462b-4e6d-a40d-1f29312e0abb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.348566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 181/354] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:40.350155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.400326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.400950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.401225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.402537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.403052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:40.403387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.404819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04585e-8dcd-4c75-800b-36bcb249328d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.422784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 174/355] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:40.424112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298c910-2d19-40cc-9e97-84926b5a490d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.424404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298c910-2d19-40cc-9e97-84926b5a490d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.425116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298c910-2d19-40cc-9e97-84926b5a490d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.426660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 175/356] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:40.427406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298c910-2d19-40cc-9e97-84926b5a490d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.491916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760e3be-c745-425f-8194-668e0cce351d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.507374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.538690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.539024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.539448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.539561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.569440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298c910-2d19-40cc-9e97-84926b5a490d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.621930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.623070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.623416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.623848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.624504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.624622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:40.624822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.625986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.626195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.638022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 176/357] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:40.639627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.642246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.761974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 182/358] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:40.763158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.775164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b2bb3-ed1e-46eb-aaf2-2ed590cff64e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.778165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.831990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995fad2-f78d-43f8-b994-5123617c3fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.832653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995fad2-f78d-43f8-b994-5123617c3fac None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.832932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995fad2-f78d-43f8-b994-5123617c3fac None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.833203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995fad2-f78d-43f8-b994-5123617c3fac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.851602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995fad2-f78d-43f8-b994-5123617c3fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3995fad2-f78d-43f8-b994-5123617c3fac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:40.857599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:40.858575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:40.859471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:40.860096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 177/359] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:40.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.861245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.861514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.887663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:40.889245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:40.897755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14647bf-8aff-4dff-b2bc-81e71d29386b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.940465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.941936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.942224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.942498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] PATH_INFO: `/v3/users/8c3e040fa002409cbfe00506b1061c8e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.943710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] RBAC: Authorizing `identity:get_user(user_id=8c3e040fa002409cbfe00506b1061c8e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.946877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.946877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 183/360] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/users/8c3e040fa002409cbfe00506b1061c8e => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:40.947771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.971927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.988244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 178/361] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:40.989135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.058935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.085852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910935-5045-4929-b6c5-0af844bfdd86 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.125149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a5ef-d574-4b58-bff1-7aa8c0b3aff9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.126664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.127539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.128044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.128513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.167414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.168050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.168324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.168592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.169873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.172184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.174280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 179/362] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.175155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.212222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.214021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.253555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.313119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d88354-1561-4e65-9d1b-6feb38e61141 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.336249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.356428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.357368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.357652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.357980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.359189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:41.359525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.360903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.361381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 184/363] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:41.364401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.372936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 180/364] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.373871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.494630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e411ada-9a0f-4eff-9eee-738b7517483d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.510367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f3db-3f45-402f-8e78-8a0e3fc44097 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.539272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.540176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.540638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.540904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.541624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] RBAC: Adding query filter params (role.id=d92a531856fd46cc87fffc7a3af54ac7, user.id=8c3e040fa002409cbfe00506b1061c8e, scope.system=all) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:41.541927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.543937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.544573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.558437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 185/365] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/role_assignments?user.id=8c3e040fa002409cbfe00506b1061c8e&scope.system=all&role.id=d92a531856fd46cc87fffc7a3af54ac7 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.558437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.567627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.570063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.570063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.570063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.570360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.571779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.573925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 181/366] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.574762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.612225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.685596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9911d-2353-4f94-93cc-f086ea505675 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.716450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340c636-a1a0-4727-b273-36a29b6cfa1b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.728030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.728685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.728966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.729238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.742138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:41.748407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:41.749558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:41.750350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 186/367] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:41.751022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.778467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.779537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.779537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.779763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.782138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.783605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.787334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 182/368] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.788504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.872038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.896909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.904920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766c50d-bd9a-4b19-8a7a-f358b6d22d83 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.949715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7e7d-0590-45c9-b660-0fc13cb8b648 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.959932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.960551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.960833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.961109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.999187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.999832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.001131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.001131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.001924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:42.002243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.006466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.010683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 183/369] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.011581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.053063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.055109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.133089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e204-e027-4236-96c2-bd5d902ce2df None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.159864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.174113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.174825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.175107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.175405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.176460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:42.176460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.177869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.179344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.185199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 187/370] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:42.187041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.190658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 184/371] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.192361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.302729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d150390-23d5-4368-8acb-20e62a80ba85 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.316401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3030ca-76a0-4db7-9e05-20e9b2540698 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.343963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.345086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.345608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.346075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.347469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.349179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.350998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 188/372] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.352137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.355600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.356908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.356908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.356908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.358010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.359526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.361171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 185/373] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.361954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.427799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.432713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.478027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15333340-30e2-4433-b3e6-776ac531d13e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.494899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85db0e8-59ad-469f-908a-41a68149b87a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.531122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.531592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.531879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.532157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.546945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.547608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.547887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.548480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.549243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:42.549583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.550981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.557773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 186/374] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.558592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.629701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.631243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.704118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86815614-415d-4694-8ba7-14e1e9f46e0b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.711301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.731693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.752309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eb160-6a76-42f8-8a82-d4a29b51c7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.752967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eb160-6a76-42f8-8a82-d4a29b51c7ce None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.753247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eb160-6a76-42f8-8a82-d4a29b51c7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.753521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eb160-6a76-42f8-8a82-d4a29b51c7ce None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.754818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63eb160-6a76-42f8-8a82-d4a29b51c7ce None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.756389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63eb160-6a76-42f8-8a82-d4a29b51c7ce None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.758316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 187/375] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.759857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04de98-5a9e-49ec-8544-0048f7dae530 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.760142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04de98-5a9e-49ec-8544-0048f7dae530 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.760415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04de98-5a9e-49ec-8544-0048f7dae530 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.761976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 188/376] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:42.763085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 189/377] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:42.769221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.769745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.770233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.770683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.797477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:42.798774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:42.882711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 189/378] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:42.884162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.893161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9aa6-523a-4d88-b465-40c7360b41d2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.933260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.934675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.935161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.935632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] PATH_INFO: `/v3/projects/0562937a351d4a8397127ac952e02625` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.937061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=0562937a351d4a8397127ac952e02625)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.938757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.942022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 190/379] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/projects/0562937a351d4a8397127ac952e02625 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:42.947628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.991943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.007761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11bac9-6f0c-4eff-ad14-c8989340b7dc None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.031615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.056152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.056769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.057049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.057322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.059189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.059390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.061296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.068369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 190/380] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.079261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0798987-05dd-49d0-a46d-66892e789cb7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.137966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.138944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.139459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.140578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] PATH_INFO: `/v3/groups/cddb04f1d4c24103b740f522438843de` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.148200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] RBAC: Authorizing `identity:get_group(group_id=cddb04f1d4c24103b740f522438843de)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.149966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.151878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 191/381] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/groups/cddb04f1d4c24103b740f522438843de => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:43.158133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.286671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.330863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19585d-cb6a-4bf0-9ea1-cdc59eef97a2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.401599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.402216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.402494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.402767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.403582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] RBAC: Adding query filter params (group.id=cddb04f1d4c24103b740f522438843de, role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=0562937a351d4a8397127ac952e02625) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.403894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.405441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.412540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 191/382] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/role_assignments?group.id=cddb04f1d4c24103b740f522438843de&scope.project.id=0562937a351d4a8397127ac952e02625&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.471351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.477374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df47da-99a7-4b34-ad3e-7ff472292150 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.517852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.518563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.518841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.519146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.534518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:43.542143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:43.544328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:43.545485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 192/383] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:43.553612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.635845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.710742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84c4db-21a5-4e02-95a7-74c1bfbb2136 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.746305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.746926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.747216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.747502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.748174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.748481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.750415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.760959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 192/384] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.767137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.938424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b661633-880b-4e0f-983c-5e3e52436955 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.014302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.014999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.015302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.015583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.019935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:44.021404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:44.023379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 193/385] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:44.029078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.209387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ca580-f166-4e2d-9b96-6f905fb050b9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.231663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:44.250174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fdf20-71c9-4b9f-9e88-1972961a3f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.250831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fdf20-71c9-4b9f-9e88-1972961a3f03 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.251113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fdf20-71c9-4b9f-9e88-1972961a3f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.251424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fdf20-71c9-4b9f-9e88-1972961a3f03 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.252817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fdf20-71c9-4b9f-9e88-1972961a3f03 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:44.254264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fdf20-71c9-4b9f-9e88-1972961a3f03 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:44.256370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 193/386] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:44.264260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.328453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:44.458144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fc900-aed2-4614-adf2-eda5ba2b8a7e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.531933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb0b3c-ae79-4bd6-9815-5da39f3edb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.532612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0b3c-ae79-4bd6-9815-5da39f3edb62 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.532893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0b3c-ae79-4bd6-9815-5da39f3edb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.533165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb0b3c-ae79-4bd6-9815-5da39f3edb62 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.535164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb0b3c-ae79-4bd6-9815-5da39f3edb62 None admin] RBAC: Authorizing `identity:create_grant(project_id=90c1eb704e8b41d8a50cece9255ebf24, group_id=aae06ebb4d1445cd8254809ff3cee84e, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:44.536654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb0b3c-ae79-4bd6-9815-5da39f3edb62 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:44.567572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 194/387] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:44 2022] PUT /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:44.748776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfc673-0bd9-4de3-bc00-66dd261d5f3e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.749539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfc673-0bd9-4de3-bc00-66dd261d5f3e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.749815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dfc673-0bd9-4de3-bc00-66dd261d5f3e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.751268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 194/388] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:44.756929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.757242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.757481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.784977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:44.786244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:44.864751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 195/389] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:45.194404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfc673-0bd9-4de3-bc00-66dd261d5f3e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.316767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dfc673-0bd9-4de3-bc00-66dd261d5f3e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.365845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260581-c8d9-4643-9f55-b087c9a67876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.366544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260581-c8d9-4643-9f55-b087c9a67876 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.366830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260581-c8d9-4643-9f55-b087c9a67876 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.367102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260581-c8d9-4643-9f55-b087c9a67876 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.381946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3260581-c8d9-4643-9f55-b087c9a67876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a3260581-c8d9-4643-9f55-b087c9a67876 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:45.388186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:45.389230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:45.391572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 195/390] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:45.397106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.422449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:45.511852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38734a3-4a50-4ab9-88be-a3310f550388 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.512160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38734a3-4a50-4ab9-88be-a3310f550388 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.512475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38734a3-4a50-4ab9-88be-a3310f550388 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.513974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 196/391] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:45.519612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.519894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.520170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.543808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cc4de-9877-42c1-9d26-c7e850ca8ca4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.556287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:45.557485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:45.603180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:45.615060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.615697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.615969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.616235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.616904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:45.617201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.618578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.639145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 196/392] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:45.647241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.696055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 197/393] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:45.752972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:45.778530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3ed4-d288-4aba-b329-cc7394bbce5c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.817454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.818081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.818356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.818627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.819807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.821243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.822800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 197/394] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:45 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:45.828381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.965471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ea012-567f-475b-82e9-2d0f1208f2a0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.017460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938d5e9-94e4-4d2e-bf5f-a971f7e115bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.018012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938d5e9-94e4-4d2e-bf5f-a971f7e115bc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.018286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938d5e9-94e4-4d2e-bf5f-a971f7e115bc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.018545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938d5e9-94e4-4d2e-bf5f-a971f7e115bc None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.019741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938d5e9-94e4-4d2e-bf5f-a971f7e115bc None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:46.021179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938d5e9-94e4-4d2e-bf5f-a971f7e115bc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:46.022789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 198/395] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:45 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:46.028475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.186406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140be713-809a-43c7-b363-542e2689aab5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.233069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e03234-ecf1-40b1-af9b-ecf693dbeefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.233791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03234-ecf1-40b1-af9b-ecf693dbeefd None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.234067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03234-ecf1-40b1-af9b-ecf693dbeefd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.234340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03234-ecf1-40b1-af9b-ecf693dbeefd None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.240542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e03234-ecf1-40b1-af9b-ecf693dbeefd None admin] RBAC: Authorizing `identity:create_grant(project_id=90c1eb704e8b41d8a50cece9255ebf24, group_id=aae06ebb4d1445cd8254809ff3cee84e, role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:46.241993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e03234-ecf1-40b1-af9b-ecf693dbeefd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:46.257489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 198/396] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:46 2022] PUT /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/e1b5b7a11157451ab80ef767124856f0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:46.270355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52acc1-a3b2-467c-9e2a-deed51f9022c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.270844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52acc1-a3b2-467c-9e2a-deed51f9022c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.270844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52acc1-a3b2-467c-9e2a-deed51f9022c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.272262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 199/397] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:46.284513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fccbf-da84-4131-9d1e-77f98288c077 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.284881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fccbf-da84-4131-9d1e-77f98288c077 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.285161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fccbf-da84-4131-9d1e-77f98288c077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.300210 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.302928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52acc1-a3b2-467c-9e2a-deed51f9022c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.321951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-864fccbf-da84-4131-9d1e-77f98288c077 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:46.324323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-864fccbf-da84-4131-9d1e-77f98288c077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:46.358011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.422188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 199/398] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:46.423531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffdaa8-9e9f-468e-bffa-1d03d618e885 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.423817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffdaa8-9e9f-468e-bffa-1d03d618e885 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.424089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffdaa8-9e9f-468e-bffa-1d03d618e885 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.425338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 200/399] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:46.430640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.431136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.431500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.456405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:46.457736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:46.469207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52acc1-a3b2-467c-9e2a-deed51f9022c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.531144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.531909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.532248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.532652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.538454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 201/400] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:46.551279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:46.556269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:46.557133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:46.558303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 200/401] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:46.564108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.635470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.682182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788dd1-42a0-478a-bd42-3e65b6b8dd25 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.717798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.718458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.718731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.718994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.719726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:46.720081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:46.721463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:46.730427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 202/402] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:46.736469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.756815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.881185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c207ba-74f9-401b-93df-c73b1e3c3441 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.899836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.916294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c57be-76cc-4a20-b1fe-e9f4964debb8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.916788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.927015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.927718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.927996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.928266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.939895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:46.945336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:46.946396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:46.947457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 203/403] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:46.948122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.963445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.964587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.965011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.965920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.967515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:46.969855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:46.971908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 201/404] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:46.977969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.062751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83d676-5ee2-433b-9c58-5bef8ffc4f8d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.098347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.099133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.099552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.099921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.104778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a590-4370-49da-82c2-e8465e14c122 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.112891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:47.118895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:47.120031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:47.120902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:47.126395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 204/405] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:47.127372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.150739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.151459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.151823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.152165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.152919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:47.153464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.154772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.165240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 202/406] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.166149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.184010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.190278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.241074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042485f-6515-4ef5-8e1c-2028d68aefa5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.279966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.282192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.283031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.283888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.286360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.286789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11ccc-d3bc-4af8-b8f0-f13d6f43a064 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.289709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.297655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 205/407] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.299947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.329492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.330248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.330529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.330803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.331494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:47.331798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.333280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.347278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 203/408] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.348564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.420273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86f9a3-e939-4ded-9991-01862f4ea81a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.441336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.447450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.465186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.465932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.466210 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.466486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.468237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.475453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.475619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf21d3-0b36-4c1a-95c7-29ffdaca11f1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.475699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 206/409] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.475699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.510384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc1ff9-3f08-4b1b-b0cd-5cc65fd155d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.511044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc1ff9-3f08-4b1b-b0cd-5cc65fd155d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.511359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc1ff9-3f08-4b1b-b0cd-5cc65fd155d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.511822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc1ff9-3f08-4b1b-b0cd-5cc65fd155d3 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.513526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc1ff9-3f08-4b1b-b0cd-5cc65fd155d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef56326c836e4e1faf0cca5b864b5d30, group_id=aae06ebb4d1445cd8254809ff3cee84e, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.515514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc1ff9-3f08-4b1b-b0cd-5cc65fd155d3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.529318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 204/410] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:47 2022] PUT /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:47.531080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d274f6-e948-414a-9264-c65c9ad298c2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.531402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d274f6-e948-414a-9264-c65c9ad298c2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.531644 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d274f6-e948-414a-9264-c65c9ad298c2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.537547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 205/411] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:47.537547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d274f6-e948-414a-9264-c65c9ad298c2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.624430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58c00f-9751-49b7-9b8d-5f9f06a89be2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.660039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d274f6-e948-414a-9264-c65c9ad298c2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.680207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee03747-531d-4406-beee-e8b7e2760cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.681476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee03747-531d-4406-beee-e8b7e2760cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.683176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee03747-531d-4406-beee-e8b7e2760cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.684252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee03747-531d-4406-beee-e8b7e2760cc7 None admin] PATH_INFO: `/v3/groups/cddb04f1d4c24103b740f522438843de` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.687729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee03747-531d-4406-beee-e8b7e2760cc7 None admin] RBAC: Authorizing `identity:get_group(group_id=cddb04f1d4c24103b740f522438843de)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.689458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee03747-531d-4406-beee-e8b7e2760cc7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.692465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 207/412] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/groups/cddb04f1d4c24103b740f522438843de => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.694055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.694508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.695035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.713643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.714292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.714570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.714843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.716012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.717450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.719140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 206/413] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.720517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:47.721045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.722318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:47.736018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.791579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.815224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 208/414] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:47.815224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.885076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f731c20-fbbf-4f57-a24f-a0ce36d4a74f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.948058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.948611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.948893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.949169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] PATH_INFO: `/v3/projects/0562937a351d4a8397127ac952e02625` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.950494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] RBAC: Authorizing `identity:get_project(project_id=0562937a351d4a8397127ac952e02625)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.953110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.957994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 207/415] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/projects/0562937a351d4a8397127ac952e02625 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.964625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.008007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72666b5e-cffb-4b87-93f4-a34b7737d402 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.072484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.072864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.073102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.073381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.080032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] RBAC: Authorizing `identity:create_grant(project_id=ef56326c836e4e1faf0cca5b864b5d30, group_id=aae06ebb4d1445cd8254809ff3cee84e, role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.081675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.083310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.102597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 209/416] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:47 2022] PUT /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30/groups/aae06ebb4d1445cd8254809ff3cee84e/roles/e1b5b7a11157451ab80ef767124856f0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:48.120103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dc36-5095-4e41-9e96-12d521c3ed59 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.144223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.174307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.175002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.175356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.175634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] PATH_INFO: `/v3/projects/0562937a351d4a8397127ac952e02625/groups/cddb04f1d4c24103b740f522438843de/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.177809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0562937a351d4a8397127ac952e02625, group_id=cddb04f1d4c24103b740f522438843de, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.179372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.204845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 208/417] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:47 2022] PUT /identity/v3/projects/0562937a351d4a8397127ac952e02625/groups/cddb04f1d4c24103b740f522438843de/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:44:48.208118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.277969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37611630-4ad8-4c0c-9d39-88a5a4d777de None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.334932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.335614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.335895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.336168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.364471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:48.369695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:48.370657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:48.372091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 210/418] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:48.377612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.508188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cf012-e93a-42f7-8c4f-039d2b049bb4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.545139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.546481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.546767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.547079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.547802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:48.548253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.549646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248344be-5fdf-4255-89e8-8522f11a88f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.555342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.559229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 209/419] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:48.566199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.681008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8706a-bf4c-488d-98c5-713f13cbb4f7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.682561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.718351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.719017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.719322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.719594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.794629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.796184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.878621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d113d0c-66dc-46bd-a905-105d76ac085b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.878946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d113d0c-66dc-46bd-a905-105d76ac085b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.879177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d113d0c-66dc-46bd-a905-105d76ac085b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.880726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 210/420] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:48.888527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.889105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.889615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.909541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:48.911059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:48.945559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 211/421] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:48.951667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.953921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.042114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 211/422] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:49.107214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ff4dd-75bd-4354-9ba3-97a5372db88d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.152789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.153429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.153702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.153967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.155122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.156573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.158193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 212/423] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.163756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.287507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9273c53-d708-44e2-86ad-95e0f72c2492 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.326681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.335760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.336401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.336675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.336943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.338053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.339505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.341363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 212/424] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.347890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.438848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.440615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.494707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059366-3c37-4c9a-8f4d-8a46dfadb2ed None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.564461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.565069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.565345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.565614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.566344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:49.566869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.568085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.583518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 213/425] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.611711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cd6e0-2de5-4fe0-a5f8-1b9d32aa5a0b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.666751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.667826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.668300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.668762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.680639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:49.686091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:49.687054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:49.687882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:49.690548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 213/426] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:49.696259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.849718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7917cf-c5e9-4aa4-a080-b69953e36690 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.893237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.893854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.894130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.894404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.895076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:49.895415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.896878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cdcb0-47df-4f9e-bd4f-26e68d960a7f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.898445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.907869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 214/427] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.914147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.979357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.032007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f66511-ba49-47de-aac2-259f3e9e99e2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.081748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.082381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.082664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.082941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.158085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.159596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.192972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04245e-3469-4d3f-96f0-7012d6f27acd None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.193276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04245e-3469-4d3f-96f0-7012d6f27acd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.193528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04245e-3469-4d3f-96f0-7012d6f27acd None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.195010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 215/428] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:50.207452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.207762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.208016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.226840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.236804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:50.238312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:50.274576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 214/429] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:50.280744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.318311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 216/430] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:50.386125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edb035-e097-418c-b2fc-206527345819 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.425715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.426362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.426643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.426919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] PATH_INFO: `/v3/projects/90c1eb704e8b41d8a50cece9255ebf24` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.428096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=90c1eb704e8b41d8a50cece9255ebf24)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.429535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.431498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 215/431] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/projects/90c1eb704e8b41d8a50cece9255ebf24 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:50.437171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.556728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd14f9-8513-4cd3-9155-17abe82cdb1d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.584671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.594879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.595572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.595854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.596127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.597353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.598870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.601057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 217/432] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:50.607303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.626712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.641506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.737117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fdf30-1219-4eae-b96b-55e30a507bf1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.756811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bbd4c-484c-45b6-bd7e-269bdff1309b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.784635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.785248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.785526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.785799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.786554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=90c1eb704e8b41d8a50cece9255ebf24) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:50.786834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.788398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31161564-a5be-4b11-8cbd-1e6e41eb08dd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.799252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 216/433] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=90c1eb704e8b41d8a50cece9255ebf24&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:50.799252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883695-162b-4960-a68a-b998428132ec None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.799252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883695-162b-4960-a68a-b998428132ec None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.799252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88883695-162b-4960-a68a-b998428132ec None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.799252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 217/434] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:50.802635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd63574-e28b-46df-886e-e5039342688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.803322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd63574-e28b-46df-886e-e5039342688d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.803603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd63574-e28b-46df-886e-e5039342688d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.803878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd63574-e28b-46df-886e-e5039342688d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.804209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.804497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.804772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.818100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd63574-e28b-46df-886e-e5039342688d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-edd63574-e28b-46df-886e-e5039342688d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:50.823721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:50.824673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:50.825464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:50.826065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:50.826065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:50.830357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 218/435] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:50.831724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd63574-e28b-46df-886e-e5039342688d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.904340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.909940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 218/436] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:50.958630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd63574-e28b-46df-886e-e5039342688d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.996675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.997288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.997567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.997841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.998510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:50.998845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.000277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.011288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 219/437] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:51.015768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.045271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.131756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65b7e6-c8f2-4a40-8d9f-9b7272c1d26c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.171178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.171817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.172099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.172371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.177320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.256427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.257900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.282241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.309597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90821728-1e3a-4d3f-9447-e32563e59da9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.326108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.345453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ca10-3757-4cde-9a0c-07c7ad34e6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.346074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ca10-3757-4cde-9a0c-07c7ad34e6ec None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.346346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ca10-3757-4cde-9a0c-07c7ad34e6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.346623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ca10-3757-4cde-9a0c-07c7ad34e6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.356378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ca10-3757-4cde-9a0c-07c7ad34e6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-fa98ca10-3757-4cde-9a0c-07c7ad34e6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:51.362263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:51.363217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:51.364041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 220/438] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:51.365178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448997e-8a04-4acd-8acc-2fc709dbd11f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.365178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448997e-8a04-4acd-8acc-2fc709dbd11f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.365456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448997e-8a04-4acd-8acc-2fc709dbd11f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.366074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 219/439] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:51.368464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 221/440] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:51.369182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.371380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448997e-8a04-4acd-8acc-2fc709dbd11f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.470702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24527eba-9609-42b9-860b-433ae3c78a19 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.472792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448997e-8a04-4acd-8acc-2fc709dbd11f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.491061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.505958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.506565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.506843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.507120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.507799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:51.508101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.509392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde45eb7-f9f3-4a0c-82a4-160711b52665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.509704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.510011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45eb7-f9f3-4a0c-82a4-160711b52665 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.510286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45eb7-f9f3-4a0c-82a4-160711b52665 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.510554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde45eb7-f9f3-4a0c-82a4-160711b52665 None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.511687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde45eb7-f9f3-4a0c-82a4-160711b52665 None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.513125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde45eb7-f9f3-4a0c-82a4-160711b52665 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.515369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 222/441] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:51.516477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.516734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.517001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.524196 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 220/442] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:51.528224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.535085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:51.536314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:51.577591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.603416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 223/443] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:51.605050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.650529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2ec20-1ff4-4de2-a372-b575a8a05660 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.688349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.688992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.689271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.689544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.690682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.692248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.693888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 221/444] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:51.699534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.716896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072e5a1-4369-4c49-9c18-790442602d23 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.752548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.753083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.753228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.753395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.753623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.798171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.812117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde6598-c1e1-4860-8d2d-54c7d814ce34 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.825562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.828153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.848409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.849034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.849301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.849655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.850307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=074c5bafaca74f698537cbd8c86e21e3, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:51.850618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.852185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.860214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 222/445] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=074c5bafaca74f698537cbd8c86e21e3 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:51.877851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.956940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 224/446] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:51.963056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.001721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba8606-3918-44e8-bfb6-8374e9c64d99 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.017854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.049860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.050515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.050791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.051066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.057226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.065473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:52.072968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:52.073889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:52.074670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 223/447] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:52.080779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.088145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae34d4-358c-44c0-afd1-e0ef4780dce2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.130869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.131547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.131829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.132103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] PATH_INFO: `/v3/projects/0562937a351d4a8397127ac952e02625` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.133255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] RBAC: Authorizing `identity:get_project(project_id=0562937a351d4a8397127ac952e02625)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.134960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.136662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 225/448] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/projects/0562937a351d4a8397127ac952e02625 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.142286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.209809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cad814-fa64-47a9-bc92-6590855563d7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.250880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.251528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.251807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.252075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.253438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:52.253635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.255339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.260035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8e38-b5d3-4b93-9932-c39a6e8fda75 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.265122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 224/449] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:52.270774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.297640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.298268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.298549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.298830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] PATH_INFO: `/v3/groups/cddb04f1d4c24103b740f522438843de` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.300001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] RBAC: Authorizing `identity:get_group(group_id=cddb04f1d4c24103b740f522438843de)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.301469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.303022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 226/450] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/groups/cddb04f1d4c24103b740f522438843de => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.305447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.308926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.321733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.376847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67391cd9-803b-4ead-a947-1971a8019956 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.411978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.412591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.412872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.413143 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.427670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767bdd-33b5-4711-b9d3-4d3d04a9921e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.461074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.461671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.461952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.462224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.462941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] RBAC: Adding query filter params (group.id=cddb04f1d4c24103b740f522438843de, role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=0562937a351d4a8397127ac952e02625) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:52.463287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.464742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.471943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 227/451] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/role_assignments?group.id=cddb04f1d4c24103b740f522438843de&scope.project.id=0562937a351d4a8397127ac952e02625&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:52.492011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.493500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.534034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.602564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 225/452] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:52.608400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.693213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.721968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10fee-3f8d-4b1d-80c1-89fb6db44fdd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.754859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.755499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.755774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.756044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] PATH_INFO: `/v3/projects/ef56326c836e4e1faf0cca5b864b5d30` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.757137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] RBAC: Authorizing `identity:get_project(project_id=ef56326c836e4e1faf0cca5b864b5d30)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.758569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.760450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 228/453] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/projects/ef56326c836e4e1faf0cca5b864b5d30 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.766585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.862002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdb6cc-6103-4306-bc18-05bb9e2d3578 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.896708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb5695-7791-4199-aa21-71c84633bd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.897327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb5695-7791-4199-aa21-71c84633bd24 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.897604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb5695-7791-4199-aa21-71c84633bd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.897876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb5695-7791-4199-aa21-71c84633bd24 None admin] PATH_INFO: `/v3/groups/aae06ebb4d1445cd8254809ff3cee84e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.898963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb5695-7791-4199-aa21-71c84633bd24 None admin] RBAC: Authorizing `identity:get_group(group_id=aae06ebb4d1445cd8254809ff3cee84e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.900422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb5695-7791-4199-aa21-71c84633bd24 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.901969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 226/454] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/groups/aae06ebb4d1445cd8254809ff3cee84e => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.908045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:53.010465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c30d9c-4f3b-4d09-b731-97ddb008953c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:53.050329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:53.050929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:53.051232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:53.051509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:53.052240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] RBAC: Adding query filter params (group.id=aae06ebb4d1445cd8254809ff3cee84e, role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=ef56326c836e4e1faf0cca5b864b5d30) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:53.053372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:53.054822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf235591-8613-46c4-a37d-3a6bf51f1fa2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:53.060566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 229/455] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/role_assignments?group.id=aae06ebb4d1445cd8254809ff3cee84e&scope.project.id=ef56326c836e4e1faf0cca5b864b5d30&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:55.731710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eedc2-56fd-45f5-96b5-5c101ef7b498 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:55.732192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eedc2-56fd-45f5-96b5-5c101ef7b498 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:55.732192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eedc2-56fd-45f5-96b5-5c101ef7b498 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:55.734275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 227/456] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:55.739834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a893a-48b1-4d36-9471-fcfc9813af9a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:55.740255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a893a-48b1-4d36-9471-fcfc9813af9a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:55.740427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a893a-48b1-4d36-9471-fcfc9813af9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:55.759871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:55.772308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ad0a893a-48b1-4d36-9471-fcfc9813af9a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:55.773533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ad0a893a-48b1-4d36-9471-fcfc9813af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:55.864064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 230/457] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:56.051473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c76c-ff9c-4403-98af-f1adf2c62170 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.051849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c76c-ff9c-4403-98af-f1adf2c62170 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.052025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c76c-ff9c-4403-98af-f1adf2c62170 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.053630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 228/458] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:56.059008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abcea5-2a96-4412-83a6-7c177dfffa00 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.059353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abcea5-2a96-4412-83a6-7c177dfffa00 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.059671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abcea5-2a96-4412-83a6-7c177dfffa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.084720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a8abcea5-2a96-4412-83a6-7c177dfffa00 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:56.086037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a8abcea5-2a96-4412-83a6-7c177dfffa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:56.177794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 231/459] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:56.255355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99278f1-a6e7-4a02-bb36-7e233a18d76e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.255673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99278f1-a6e7-4a02-bb36-7e233a18d76e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.255926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99278f1-a6e7-4a02-bb36-7e233a18d76e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.257512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 229/460] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:56.263092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973a2b2-0e7f-44e1-bcff-589c74a3455e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.263428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973a2b2-0e7f-44e1-bcff-589c74a3455e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.263707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973a2b2-0e7f-44e1-bcff-589c74a3455e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.288432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3973a2b2-0e7f-44e1-bcff-589c74a3455e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:56.289634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3973a2b2-0e7f-44e1-bcff-589c74a3455e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:56.301849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99278f1-a6e7-4a02-bb36-7e233a18d76e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.305668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.407610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.419227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 232/461] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:56.468850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99278f1-a6e7-4a02-bb36-7e233a18d76e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:56.494873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9db18c-0e89-4a05-b5c8-047b84a7d95c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.495154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9db18c-0e89-4a05-b5c8-047b84a7d95c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.495621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9db18c-0e89-4a05-b5c8-047b84a7d95c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.497337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 233/462] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:56.502656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.503248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.503775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.518465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:56.519106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.519415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.519694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.543012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:56.544833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:56.547270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:56.549234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:56.565900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:56.566824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:56.567704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:56.567704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:56.567704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:56.569906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 230/463] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:56.570918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.635236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.651240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 234/464] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:56.651240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.711890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228657af-5ff1-4175-a1f8-1a2214486a94 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:56.750265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:56.750995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.751244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.751497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.764786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bec05-cb0f-40cd-a885-f3a366fe4005 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:56.766410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:56.771852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:56.791759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:56.792668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:56.793497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:56.793497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:56.793497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:56.793669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 231/465] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:56.794797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.795396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.807413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:56.808324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.808796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.809259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.810111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:56.810604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:56.812740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:56.835727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 235/466] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:56.836866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.922875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cf59f-de86-4640-8db4-a2a3b0983e09 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:56.933856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.951113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc99aaa-5b3c-4af6-a83c-e54c1de9396d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:56.968100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:56.968792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.969260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.969554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.984231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:56.985699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:56.987751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:56.988360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.988633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.988898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.989528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:56.989828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:56.991179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.002883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 236/467] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:57.003711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:57.005089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:57.005983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:57.006806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:57.006806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.006806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:57.006806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 232/468] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:57.007448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.063591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.114668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fa545-2655-4cc6-8eb3-b2515b93959d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.128196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a6b4c-312e-4f20-9846-d97a97e7949f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.153632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.154268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.154558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.154836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.163121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.163751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.164032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.164305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.165221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.166392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.167402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.168868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:57.187820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:57.188784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:57.189607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:57.189607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.189607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:57.189607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 233/469] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:57.190388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.193306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 237/470] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:57.195634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.207844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.316958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59169ac-c899-4d03-b38a-454849e94f0b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.335091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1db3d-e326-4f27-a8aa-cd868c34f215 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.337072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.371448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.372647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.373328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.374138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.375121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:57.375463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.376860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.379718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455497f5-4248-4367-8a54-08444743d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.380591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455497f5-4248-4367-8a54-08444743d202 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.380591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455497f5-4248-4367-8a54-08444743d202 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.381221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455497f5-4248-4367-8a54-08444743d202 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.381506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455497f5-4248-4367-8a54-08444743d202 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.383013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455497f5-4248-4367-8a54-08444743d202 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.391889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 234/471] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:57.392748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.411006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 238/472] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:57.412262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455497f5-4248-4367-8a54-08444743d202 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.519574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.558961 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f29da-2af6-487d-acce-3c0d6c182198 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.587510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455497f5-4248-4367-8a54-08444743d202 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.607275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.607950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.608227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.608499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.609150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:57.609454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.610893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca1b59-bd2b-4e93-a88d-b3d29f3424ad None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.627052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 235/473] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:57.632513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13cd4a-cf81-4bec-bb50-c00060f19131 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.632793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13cd4a-cf81-4bec-bb50-c00060f19131 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.633079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13cd4a-cf81-4bec-bb50-c00060f19131 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.635354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 236/474] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:57.636874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13cd4a-cf81-4bec-bb50-c00060f19131 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.638201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab06ee-25d4-4773-9dbc-81bb0929c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.639726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab06ee-25d4-4773-9dbc-81bb0929c269 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.642034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab06ee-25d4-4773-9dbc-81bb0929c269 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.642598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab06ee-25d4-4773-9dbc-81bb0929c269 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.643502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab06ee-25d4-4773-9dbc-81bb0929c269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.645310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab06ee-25d4-4773-9dbc-81bb0929c269 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.676698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 239/475] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:57.677924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.678493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.678493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.701122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:57.703852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:57.718084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.801379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 240/476] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:57.821847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13cd4a-cf81-4bec-bb50-c00060f19131 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.857817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.864130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.864756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.865035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.865309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.865967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.867578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.896583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 237/477] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 249 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:58.103748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.222562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbedb2-b638-4882-845b-d5002d02d16e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.257834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.258506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.259034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.259336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.260019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:58.261378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:58.270012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.sql.core [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 14:44:58.270012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 14:44:58.270012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [parameters: {'id': 'e970beea4c2f444aa38d78dfcb7e3f32', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 14:44:58.270012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 22 14:44:58.286734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return ref.to_dict() Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:44:58.287711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application yield resource Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise self.value Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application session.commit() Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 14:44:58.289111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.flush() Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application compat.raise_( Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 14:44:58.290208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:44:58.291107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:44:58.292050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [parameters: {'id': 'e970beea4c2f444aa38d78dfcb7e3f32', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:58.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 14:44:58.295602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:44:58.297444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 241/478] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Sep 22 14:44:58 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:58.297532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.447880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acce23d-8eb0-4a55-b8be-e970a736a548 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.521028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb026-1b2a-4219-a772-f6745fcee657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.521696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb026-1b2a-4219-a772-f6745fcee657 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.521980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb026-1b2a-4219-a772-f6745fcee657 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.522261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb026-1b2a-4219-a772-f6745fcee657 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.537392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcb026-1b2a-4219-a772-f6745fcee657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-97fcb026-1b2a-4219-a772-f6745fcee657 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:58.542823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:58.545111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:44:58.547120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 238/479] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:58.554392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.572512 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:58.707296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc876d-78b8-40d6-92b4-5faa494aa994 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.754072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.754615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.754892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.755165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.755884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:58.756190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:58.757605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3007db-663f-44fb-89b4-4e0805bf1d6c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:58.772259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 242/480] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:59.741084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2aa690-cb42-4662-b5ab-cc74c8fbf674 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:59.741624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2aa690-cb42-4662-b5ab-cc74c8fbf674 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:59.741624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2aa690-cb42-4662-b5ab-cc74c8fbf674 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:59.743193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 239/481] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:59.751877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443541-ee41-4de1-8512-f5e57bf4a76b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:59.752383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443541-ee41-4de1-8512-f5e57bf4a76b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:59.752802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443541-ee41-4de1-8512-f5e57bf4a76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:59.784799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-55443541-ee41-4de1-8512-f5e57bf4a76b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:59.786118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55443541-ee41-4de1-8512-f5e57bf4a76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:59.882946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:59.888682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 243/482] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:00.236659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2aa690-cb42-4662-b5ab-cc74c8fbf674 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.270354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2a4f-ffa1-4560-9932-1f3bef0560ea None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.270682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2a4f-ffa1-4560-9932-1f3bef0560ea None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.270915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2a4f-ffa1-4560-9932-1f3bef0560ea None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.272635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 244/483] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:00.281105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.281407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.281656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.314035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:00.315265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:00.325905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:00.406216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2aa690-cb42-4662-b5ab-cc74c8fbf674 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:00.453338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:00.453980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.454253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.454518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.465702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 245/484] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:00.468305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:00.474079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:00.474912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:00.476079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 240/485] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:00.481032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.657740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0cc21-43d8-4e84-ad71-b2ccd548d703 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:00.706464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:00.720580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:00.721508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.721968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.722421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.723467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:00.724235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:00.725816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54df41-db11-45c8-8f77-a9214ea407fe None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:00.735951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 246/486] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:00.741309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.749113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9bb9d-7ae8-461b-87d6-cd700464a8a6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.749640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9bb9d-7ae8-461b-87d6-cd700464a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.749814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9bb9d-7ae8-461b-87d6-cd700464a8a6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.751589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 247/487] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:00.757259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94723c-d1c2-48fa-b5fc-b9b7f049fd01 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.758224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94723c-d1c2-48fa-b5fc-b9b7f049fd01 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.759108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94723c-d1c2-48fa-b5fc-b9b7f049fd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.797617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ee94723c-d1c2-48fa-b5fc-b9b7f049fd01 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:00.799291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ee94723c-d1c2-48fa-b5fc-b9b7f049fd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:00.873160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd34c7-a180-40af-b56a-e87c72d4c4b2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:00.879982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 248/488] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:00.884757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb8e84-4553-4223-8a1e-06965bcc2f48 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.885041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb8e84-4553-4223-8a1e-06965bcc2f48 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.885311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb8e84-4553-4223-8a1e-06965bcc2f48 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.886730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 249/489] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:00.890921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb8e84-4553-4223-8a1e-06965bcc2f48 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.892415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:00.930738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:00.931683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.932072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.932451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.950006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:00.951956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2e534480-e010-4ac0-9357-ddac944666a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:00.975252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:00.976247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:00.977087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:00.977087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:00.977087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:00.977087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 241/490] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:00.979482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd508-de72-4931-a019-de519add4089 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.979845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd508-de72-4931-a019-de519add4089 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.980269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd508-de72-4931-a019-de519add4089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.007915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2b5bd508-de72-4931-a019-de519add4089 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:01.010307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2b5bd508-de72-4931-a019-de519add4089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:01.018589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb8e84-4553-4223-8a1e-06965bcc2f48 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.055545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.074174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.075331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.075619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.075889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.095242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:01.095846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 242/491] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:01.096105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.096978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:01.097774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 250/492] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.099848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd508-de72-4931-a019-de519add4089 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.105202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.210133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.222028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd508-de72-4931-a019-de519add4089 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.260366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.261254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.261944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.262364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.263662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.264452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.266215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.269838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8301d25-0b08-4544-8c77-6cb178bd8eac None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.280623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 243/493] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:01.281486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.324175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.324835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.325121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.325385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.326053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.326393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.327866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.336992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 251/494] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:01.337833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.399370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.415818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da13afc-4517-4262-8642-44522b6f9ef1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.456820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.457480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.457757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.458026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.470213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:01.475464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.476343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.477151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:01.477779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80769b1a-2884-496d-9e84-61697d9b5d8b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.477858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 244/495] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.477858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.492794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.522419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.523170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.523372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.523630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.532240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.533688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:01.546880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.547874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:01.548763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 252/496] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.549339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.593064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24addf6a-02ef-44a0-bf72-2909990eb8d1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.631651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.632309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.632589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.632856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.645494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.647836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:01.664671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.665643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:01.666473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:01.666473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:01.666473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:01.666473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 245/497] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.666687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.673923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6934-558d-406c-ac6e-bbdaf4d4fa26 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.676477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.718567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.719275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.719729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.719841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.729302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.732887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:01.738177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.739250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 253/498] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.740168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.773600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a7ef3-7f2d-4411-823c-93ba269e69b1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.806663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.807300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.807577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.807848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.808503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.808802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.810165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.817998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 246/499] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:01.818792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.840803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a2063-742b-481a-8948-62cd9e529ef6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.877653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.878254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.878537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.878812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.879571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.879878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.881323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e127b-5b95-4880-950e-81f71808ca7f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.918260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 254/500] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:01.919549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce50c3-e0a4-47bc-beff-fd6bdbcf8da3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.919840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce50c3-e0a4-47bc-beff-fd6bdbcf8da3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.920116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce50c3-e0a4-47bc-beff-fd6bdbcf8da3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.921322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 255/501] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:01.922012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce50c3-e0a4-47bc-beff-fd6bdbcf8da3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.927012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b1332-cc0d-4729-aeb1-8f272e7b55f0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.945982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.950254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.962513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.963113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.963420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.963691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.964739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.965042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.966401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.977902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 247/502] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:01.978708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.024216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce50c3-e0a4-47bc-beff-fd6bdbcf8da3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.057794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.058388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.058663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.058936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.059617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.059916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.061813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.069691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 256/503] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.070444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.078563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9376ea-d306-4b48-9825-0df7bd6dddd6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.111966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.112726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.113049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.113362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.123780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.125223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-5bc330e2-8cd4-48c8-b07a-a61ad45d68e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.140201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.141145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.141973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.141973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.141973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:02.141973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 248/504] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.142441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.142694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.142937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.158926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.160310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:02.161587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:02.176384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ff31-c62f-4a19-98c2-9a62b3eeb198 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.190358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.212356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.212970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.213254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.213525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.229583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 249/505] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:02.230429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.297013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.298501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.339294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff22aa-63cd-45a6-9880-2a2ec8027f30 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.376313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.377302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.377793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.378232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.380019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.388154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.390217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.401934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 257/506] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:02.401934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:02.403823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.405469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 250/507] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.406339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.436657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.509496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b7dd-a3be-435b-9df2-fbbcce0f3e8b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.518795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288d7c9-b66b-4023-a125-5ed885da3437 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.545797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.546416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.546691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.546956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.552591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.553530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.553825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.554104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.555126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.555475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.556657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.556866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.558187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.569585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 251/508] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.570384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.572215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.573115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.573974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.573974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.573974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:02.573974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 258/509] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.574194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.613751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.663340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.679269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1111bd6-179a-45a6-a6b2-fb92b07cf60c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.679428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d01d16-d92c-4781-9d4a-15ef4f332a67 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.713552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.714122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.714404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.714677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.715435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.715529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.715743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.716105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.716385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.716659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.717193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.727519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.728949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.734771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 259/510] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.736025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.743533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.744613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.745453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.745453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.745453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:02.745453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 252/511] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.745453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.832860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.840244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e249abb-6044-4d3e-bd16-62f57df85a5e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.845051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910ab5-f77c-42ac-a0e0-4584bda9ec23 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.874444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.875624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.876119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.876621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.878838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.880118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.880621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.881120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.890484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.892496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.893222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.893970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.895976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:02.908798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.910025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:02.910931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 253/512] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.911517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.912309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.913120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.913120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.913120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:02.913305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.913378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 260/513] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.913378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.013236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791e23f-5cfd-4ce0-99da-22ca51676842 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.016323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fda885-14dc-4552-a252-01ce4578f1b4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.046637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.047256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.047535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.047807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.049293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.049895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.050180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.050455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.051091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.051428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.052828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.062875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.065634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 254/514] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.066821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.118043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.119508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.126557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.170327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ae66-0888-40f1-9a78-ea7bf295c8b7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.205349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.205954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.206232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.206513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.207142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.207474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.208836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.220395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 255/515] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.221216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.226904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 261/516] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:03.226904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.277068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.325618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4122d35-83bb-49be-a2b4-dc91e50ae3fc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.331663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d0505-a857-46ad-b0ba-2bd3ac1c11d5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.343230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.360484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.361092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.361371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.361645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.362280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.362605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.363992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.365550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.366151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.366424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.366710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.367445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.367751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.369236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.381978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 256/517] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.382837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.390006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 262/518] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.390719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.487630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4d0ad-af62-45c6-9880-460236952eae None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.490302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eb4af-f695-48a9-8a4d-d1aeaff1ac6f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.493395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.522511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.523142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.523451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.523717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.527999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.528623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.528905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.529179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.530815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.532281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.536068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.537491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:03.543631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.544559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 257/519] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.546046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:03.549341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.550319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 263/520] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.551790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.592937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.651431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b457de-438c-493a-9da9-b29144083b1a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.652781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b55ee-e778-4a3a-a32e-55b7dbba6b3d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.683732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.684334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.684608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.684876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.684967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.685525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.686095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.686398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.686668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.687458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.697073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.698493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.709580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 264/521] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:45:03 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:03.710414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:03.713122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.714132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:03.715004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:03.715004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:03.715004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:03.715004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 258/522] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.715004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.751221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.825501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.837414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0da33-db7e-46c5-bbf2-0e81069c33a7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.846681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25481c-923c-45b9-aa80-1b717430ee7b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.878435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.879028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.879409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.879683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.880427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.880735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.882197 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.887151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.887815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.888098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.888370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.924278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 259/523] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.925422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.968720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.970206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.991788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.042194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a737fa75-c92c-401a-8890-7f15aed3a278 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.080132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 265/524] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:04.080132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.083333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.083957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.084235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.084507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.085198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.085501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.087406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.107732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 260/525] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.108571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.120441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.194611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a847c54-8cd4-4c6a-9e36-7049faeaf7cc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.214625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747161cd-598c-4170-aa34-a4a56658c286 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.220357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.235895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.236739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.237180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.237447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.238097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.238405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.239836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.252969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 266/526] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.253813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.262089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.262708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.263433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.263722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.337967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.339638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.350634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.359079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6eb22-c5f3-4a57-962b-a2a9af9a438f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.394908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.395561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.395849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.396123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.407384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.408840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:04.424049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:04.425255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:04.426832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:04.426832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.426832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:04.426832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 267/527] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:04.426832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.459875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 261/528] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:04.460626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.466575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.545656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbd2b9-2d16-44cd-933a-5805900be9f8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.584998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd854c7-0d06-4a52-876b-5406878043d9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.586455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.587410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.587815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.588209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.589042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=ff1a6a40166e4f51a14cca9cd510e680) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.589458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.591006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.596789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.607246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 268/529] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/role_assignments?user.id=ff1a6a40166e4f51a14cca9cd510e680&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.607246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.645283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.646071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.646425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.646768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.656659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.659144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:04.679931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:04.681016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:04.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 262/530] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:04.682362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.739590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.749225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fd3c6-8142-4d74-b3a5-5a0f425ab62e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.798602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.799238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.801406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.801406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.801406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.801406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.802775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.820336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 269/531] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.820336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.828488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d69b2-1e10-4fb1-b972-7d3c249ee0fd None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.866315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.867514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.868007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.868841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.881963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.884985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:04.905305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:04.906259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:04.907115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:04.907115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.907115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:04.914504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 263/532] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:04.914504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.939654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.943762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb20e39-1dc8-4126-b8ad-2e77afb4035e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.991601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.993042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.993837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.994316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.995402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.996722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.998497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.022991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 270/533] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.024216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.030586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15232949-eda8-4c6c-a2cb-0d5c1523207b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.049094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.066055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.066699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.066977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.067293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.074267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.075760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:05.088560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:05.089633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 264/534] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:05.090503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.172989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90a694-429a-4982-be68-784fd076aa80 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.194782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.211731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.212345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.212625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.212899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.213535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.213842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.215415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.223157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47670c4-9597-4cf1-a35d-c71e6605dc37 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.231555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 271/535] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.233172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.269505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.270704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.271678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.271862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.272902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=623f9046c3924b35b36d8807f8eef54c) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.275415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.278223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.285651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 265/536] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/role_assignments?user.id=623f9046c3924b35b36d8807f8eef54c&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.286632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.334134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.394591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a3f1-6058-40ae-9aee-f7283a5e73db None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.422922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9732f-85d2-481b-8e37-5ac42789d942 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.468923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.471643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.473782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.474315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.490911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.492422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.497306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.497925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.498240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.498581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.499213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.499541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.500973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:05.520698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:05.521564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.522390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:05.524420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 266/537] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:05.527864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 272/538] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.534146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.536340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.539588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.647620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67f218-1510-4031-b7c2-103026b7161b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.700906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.702082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.702368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.702640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.703360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.703670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.705103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.733600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 267/539] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.735709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137009bd-ad80-420b-b107-49918abd90d6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.740595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.788636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.789163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.789442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.789716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.790439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=60c809b5a47a4983be26116553934387) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.790747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.792340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db9ac9-c948-4cf9-80e9-acf97ae00103 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.795671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.803347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 273/540] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/role_assignments?user.id=60c809b5a47a4983be26116553934387&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.832084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.872743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540277e-87f7-4a80-a232-601771b1cb67 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.910623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.911306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.911537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.911792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.912526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=b4e09c957a104f2299ab55d8283124c5) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.912831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.914276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09071d0d-93e4-47ff-b4ce-a6128e2e25d3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.921429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 268/541] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/role_assignments?user.id=b4e09c957a104f2299ab55d8283124c5&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:06.258100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b5f20-1b7c-42d9-b1e2-cd0246175281 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.259237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b5f20-1b7c-42d9-b1e2-cd0246175281 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.259684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b5f20-1b7c-42d9-b1e2-cd0246175281 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.261436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 274/542] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:06.267701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.268245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.268716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.296643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:06.297906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:06.381388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 269/543] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:06.762929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b5f20-1b7c-42d9-b1e2-cd0246175281 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:06.880652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b5f20-1b7c-42d9-b1e2-cd0246175281 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:06.924408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079baa-4cce-4a1d-8c56-6d5d15f1cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:06.925085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079baa-4cce-4a1d-8c56-6d5d15f1cad8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.925363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079baa-4cce-4a1d-8c56-6d5d15f1cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.925684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079baa-4cce-4a1d-8c56-6d5d15f1cad8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.938908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3079baa-4cce-4a1d-8c56-6d5d15f1cad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a3079baa-4cce-4a1d-8c56-6d5d15f1cad8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:06.943828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:06.944700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:06.945516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:06.949954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 275/544] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:06.951411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.038153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.082258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05e731-5491-470a-a53c-24c3e60bfec4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.090038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90af6c-ef5d-41ae-96e6-dec1e63ed458 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.090369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90af6c-ef5d-41ae-96e6-dec1e63ed458 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.090711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90af6c-ef5d-41ae-96e6-dec1e63ed458 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.096586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 276/545] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:07.098203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.098489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.098766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.119122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:07.120470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:07.145084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.146142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.146645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.147122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.148073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:07.148392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:07.149854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:07.167232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 270/546] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:07.172234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.194632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.221608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 277/547] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:07.353429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181b185-5805-49da-9194-469c5a6e5e20 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.385380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.410624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.411274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.411554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.411827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.424571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:07.426020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:07.444751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:07.445693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:07.446928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:07.446928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:07.446928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:07.446928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 271/548] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:07.453477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.604628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8cda-7d58-45ff-89c5-fc71199c5d49 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.658673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.659356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.659641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.659915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.660563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:07.660868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:07.662245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:07.677634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 278/549] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:07.683432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.691484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.753670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.831789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368a0ef-883a-4063-a6d2-6f374ed2cbf0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.915179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940fff5-7c7a-404f-953c-55ccda7cb63b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.917734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919dc7d-6e4d-4f92-9013-257d8deda368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.918458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919dc7d-6e4d-4f92-9013-257d8deda368 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.918996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919dc7d-6e4d-4f92-9013-257d8deda368 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.919305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919dc7d-6e4d-4f92-9013-257d8deda368 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.949120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919dc7d-6e4d-4f92-9013-257d8deda368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a919dc7d-6e4d-4f92-9013-257d8deda368 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:07.961175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:07.962310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:07.963441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:07.967281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 279/550] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:07.970014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30aee6-a9d2-492a-93ae-4888f28d405c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.970299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30aee6-a9d2-492a-93ae-4888f28d405c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.970578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30aee6-a9d2-492a-93ae-4888f28d405c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.972166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 280/551] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:07.972889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30aee6-a9d2-492a-93ae-4888f28d405c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.995002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.996313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.996873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.997409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.006521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.008558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-579cb30e-d25d-40bf-be32-e486d562a3ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:08.021846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.022994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:08.024297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:08.026688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 272/552] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.028489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.029023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.030008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.050943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:08.052544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:08.083175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.088207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30aee6-a9d2-492a-93ae-4888f28d405c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.099677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.140723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.141353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.141630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.141921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.142571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:08.142879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.144340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.152645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 273/553] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:08.153507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 281/554] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:08.154022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.168692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.272510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2da92-1ea9-489a-8bc0-0aaddce34438 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.322794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.323864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.324347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.324835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.325769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:08.326987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.328677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.352207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 274/555] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:08.363233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.363387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d7a3d-8ffd-43a0-a88d-6e7efca00eaa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.405047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.443944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.444516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.444795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.445072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.482689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.484219 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:08.510907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.512164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:08.513228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:08.513228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:08.513228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:08.517711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 282/556] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.521885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.546737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b607dab-5529-4694-bf3c-85d958faed7b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.564333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.586433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.587394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.587822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.588092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.652525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32501ca-05a3-4c48-b83e-99a22de18c78 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.671428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.672878 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.686995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.687640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.687918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.688193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.688829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:08.689134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.690509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e4d4a-2d00-4bc2-94a6-12e24cf950e7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.693238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.702892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 283/557] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:08.704040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb943a-5c29-49b8-822c-e47a9bb262f6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.704331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb943a-5c29-49b8-822c-e47a9bb262f6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.704608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb943a-5c29-49b8-822c-e47a9bb262f6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.705961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 284/558] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:08.706826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81fd2-5cae-45a3-a455-d053c73b2ec2 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.707162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81fd2-5cae-45a3-a455-d053c73b2ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.707624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81fd2-5cae-45a3-a455-d053c73b2ec2 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.708719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 285/559] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:08.709424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81fd2-5cae-45a3-a455-d053c73b2ec2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.779548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 275/560] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:08.780367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.793050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.811301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81fd2-5cae-45a3-a455-d053c73b2ec2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.844469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338f77e-02bc-4b33-bd81-dee07636cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.845091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338f77e-02bc-4b33-bd81-dee07636cf54 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.845363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338f77e-02bc-4b33-bd81-dee07636cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.845630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338f77e-02bc-4b33-bd81-dee07636cf54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.855261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338f77e-02bc-4b33-bd81-dee07636cf54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a338f77e-02bc-4b33-bd81-dee07636cf54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:08.860522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.861564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 286/561] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.865185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.865185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.880963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce373ce-9d85-47d3-8d72-9d6792ef4ff8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.881846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:08.883043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:08.903851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.917930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.918589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.918863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.919133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.926162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.927631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-02dc5e2c-41f0-430e-b928-9a21b9ba8cb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:08.939413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.940440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:08.941398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 276/562] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.941989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.942217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.942491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.954248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 287/563] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:08.955039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.970813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:08.972790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:09.036011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.046852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 277/564] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:09.060154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.063495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835ef9f-5cda-4ddc-a59b-a5ac6477792c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.099285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.099920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.100199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.100467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.112049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.113538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:09.129569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.130547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:09.132045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:09.132045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.132045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:09.132045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 288/565] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.132045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.190470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5acbb-2754-4eb3-b9d6-d7eeb609c8e5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.203868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.230471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.231001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.231307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.231579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.232762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.233075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.234465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.243078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 278/566] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.244098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.264201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9aee1-4cdd-491b-b13f-16abf32f3ca9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.294282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.304088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.304801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.305154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.305489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.306212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.306521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.307997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.327386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 289/567] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.328217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.356967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43792a4d-527d-4db2-88c4-9cff3aa43160 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.391865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.392503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.392803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.393098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.393799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.394107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.395597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.407935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 279/568] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.408741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.428301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd56b-dac5-49ba-b13d-0ac0eedec5e6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.438802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.460928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.461546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.461820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.462087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.472462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:09.477238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.478166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 290/569] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.479288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.518409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce50ee-0f1c-49f9-b9d1-6fcc7c74b5c8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.526587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.552525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.553100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.553377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.553647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.563409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.564839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:09.579332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.580332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:09.581231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:09.581231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.581231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:09.581398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600d47f-1529-4ecf-bde7-e5c40dba41bd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.581471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 280/570] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.581471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.619310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.619937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.620221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.620496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.630811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:09.636137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.637215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 291/571] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.638055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.665596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.682680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c2fff-dc19-40ad-8e61-77b7f5f451c7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.721351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.721970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.722240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.722622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.733641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.735092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.748109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051a739-dbea-4794-8ee8-e64de7d94f51 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:09.749365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.750580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:09.751537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:09.751537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.751537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:09.751537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 281/572] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.751771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.758257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.788262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.788772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.789054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.789326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.796886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.799136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:09.811311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.812329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:09.813209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 292/573] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.813691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.865076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc656900-2c4e-41fd-ac0b-cb4f7b088c60 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.899830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.902032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.902734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.903322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.904903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.905410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.907721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.911427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.919562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 282/574] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.920403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.936298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6b5ec-d0fa-485f-9647-81e1a40bac0b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.971717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.972317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.972591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.972857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.973490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.973793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.975157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.986245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 293/575] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.987154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.014387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.025938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7f5f-8c16-4854-b504-9ce123a2662d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.059335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.059978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.060198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.060468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.061120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.061419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.062781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.070499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 283/576] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.071309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.088141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b1917-b814-4443-97bb-6e14a5fda43f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.123941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.124587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.124893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.125170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.125925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.126265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.127805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.133974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.140683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 294/577] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.146460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.176530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a80b32-e2b3-4f87-92b2-1176aa0d804c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.210145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.210745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.211023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.211324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.212008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.212375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.213738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.225248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 284/578] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.226047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.230947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.245990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bcfda-2160-4cd1-9998-385405f71a87 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.281556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.282187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.282467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.282742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.292621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.294066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:10.308463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.309658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:10.311165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:10.311165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.311165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:10.311165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 295/579] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.311165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.333975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6a25d-225f-4ae0-b55a-180c3a32e406 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.353936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.368824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.369450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.369726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.369991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.377392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.378829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:10.390954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.396662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 285/580] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.397857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.423879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d5bf1-8eae-41ea-a5ad-d52605f889d5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.456940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.457566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.457849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.458123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.460776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.469797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.471329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:10.486382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.487513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:10.488415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:10.488415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.488415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:10.488415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 296/581] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.488415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.501532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e85f8-9762-425c-96cf-91541c43ac7f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.535639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.536262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.536541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.536819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.543321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.544791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:10.556022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.557107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 286/582] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.558485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.591184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd1868-f20b-48f6-a1d3-f0bacfa70240 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.604335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.625278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.625884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.626164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.626438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.627166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=543542d17db0495f8151e779e6c6218f) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.627509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.628949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.634573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 297/583] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/role_assignments?user.id=543542d17db0495f8151e779e6c6218f&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.635393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.675933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531b0e4-6767-4511-9036-44e0a530a232 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.704221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.720851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.721475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.721766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.722031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.722760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.723212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.724772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.744082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 287/584] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.744082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.762668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678ab54-62f3-49fd-9269-8ca4fcdc8d75 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.801193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.802162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.802534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.802784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.803507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.803815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.805268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.824579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 298/585] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.825426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.864780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.867699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984139fa-cb99-4157-8146-71438b6a9d69 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.906417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.907031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.907347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.907621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.908261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.908564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.909948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.923594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 288/586] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.924742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.940941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe605e4c-563c-457f-9d9d-d9711691bcf1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.952572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.978400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.979065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.979378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.979652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.980646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.980765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.982197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.000334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 299/587] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.000334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.037165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028de55-2327-4dbd-bd69-1c1d3c8d9c86 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.071445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.072168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.072453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.072730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.080531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.081986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:11.094099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.095304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:11.096473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:11.099870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 289/588] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.100725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.111521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cabb1d-deef-4170-9fa8-c5b31d67fedb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.125462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.150391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.151193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.151611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.151972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.162877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.164361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:11.178472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.179782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:11.180683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:11.180683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.180683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:11.180683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 300/589] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.180683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.210338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.221523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba777d12-5c9f-478c-94b0-116e6f081727 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.257345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.257975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.258259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.258531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.267641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.269440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:11.281319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.282507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:11.283493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 290/590] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.284087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.297870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cff0a-2e54-413d-a65b-cd03922cb8a2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.336390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.337048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.337332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.337609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/ff1a6a40166e4f51a14cca9cd510e680/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.352452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=ff1a6a40166e4f51a14cca9cd510e680, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.353901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.367582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 301/591] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:11 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/ff1a6a40166e4f51a14cca9cd510e680/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:11.368654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.393419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.421957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5de0f-91b0-44f5-bc3e-9d0e5421de26 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.461546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.462174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.462547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.462829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.463601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.463912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.465379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.484783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.495170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 291/592] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.496282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.542098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022d99f-0e1c-41c9-b7a2-8c0422c47e38 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.586913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.587744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.588095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.588442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.589163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.589539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.590987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.603628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 302/593] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.604523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.630192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8f5c7-2d0a-40a9-9c9e-f56bf396f69a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.681459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.682683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.683043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.683678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.684495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.685119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.688139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.698336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.713033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 292/594] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.714125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.723000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52d02f-d27e-4c8f-b4bd-482d50b6ceba None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.756394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.763123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.764678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.765160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.765629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.777822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.780338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:11.797595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.798763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:11.799900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:11.799900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.799900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:11.801763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 303/595] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.802706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.852478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef40e-a771-4abd-a598-4a190c59470d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.894663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.895962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.896391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.896811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.904972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.906839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.919322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309deb9f-2b8b-4eb4-bda9-5708c12463e9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:11.921540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.922576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:11.923706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 293/596] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.924289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.973042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.974137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.974743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.975377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.994479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.997376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.021735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:12.025643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:12.027834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:12.029819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:12.029819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:12.029819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:12.033524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 304/597] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:12.035436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.097461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.108385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad031085-9128-4fb2-9946-8b5782c4b7c8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.168342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.169359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.170381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.170863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.171986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.172509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.174862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.201991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 294/598] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.201991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.222801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378d472-6cb4-42cb-be1f-d74cccaa5476 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.270153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.270681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.270961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.271263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.271963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.272270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.273927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.286754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 305/599] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.287829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.325709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30624f-e8c2-46a3-b996-fd358208c485 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.352425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.373787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.373996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.374253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.374494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.375194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.375535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.376980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.391090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 295/600] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.392116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.408837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.418310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f11cb-c1e1-46a2-a85b-0af23528e6b4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.468793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.469858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.470601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.471061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.481915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.486007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:12.505816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:12.506795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:12.514383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 306/601] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:12.514383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.519743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51406c-582c-4ef2-be36-26630994ca12 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.556247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.556899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.557236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.557640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/623f9046c3924b35b36d8807f8eef54c/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.576688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=623f9046c3924b35b36d8807f8eef54c, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.578212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.603925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 296/602] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:12 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/623f9046c3924b35b36d8807f8eef54c/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:12.605755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.644547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.723788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a289e92-ba66-46b8-aa54-ffd367a28d26 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.747233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd07b1-1d71-447f-b030-f11ba29236c7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.796434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.797212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.797499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.797776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.798520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.798828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.800814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.816152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 297/603] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.817023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.817740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.818083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.818422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.823266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.836548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.838126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:12.858701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:12.859864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.860966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:12.863016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 307/604] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:12.869043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.891987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.940257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.968206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c99a-d623-40b7-aee6-28cda2c620d5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.007535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2914cf-635e-4d07-bf4d-775515f0500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.008287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2914cf-635e-4d07-bf4d-775515f0500c None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.009070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2914cf-635e-4d07-bf4d-775515f0500c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.009179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2914cf-635e-4d07-bf4d-775515f0500c None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/b4e09c957a104f2299ab55d8283124c5/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.012346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457677cf-966a-4fff-a5b8-468a30d57b4a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.027524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2914cf-635e-4d07-bf4d-775515f0500c None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=b4e09c957a104f2299ab55d8283124c5, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.029758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2914cf-635e-4d07-bf4d-775515f0500c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.047821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 298/605] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:12 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/b4e09c957a104f2299ab55d8283124c5/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:13.049599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018276-c94c-4d01-a864-530125d6a900 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.049896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018276-c94c-4d01-a864-530125d6a900 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.050276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02018276-c94c-4d01-a864-530125d6a900 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.057272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 299/606] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:13.059304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.059588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.059925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.060668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.061518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.062016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.062490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.063473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:13.064006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.065635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.085252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 308/607] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:13.085449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:13.090468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:13.092988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.211859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 300/608] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:13.250185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:13.279833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2dc2b-b329-489a-a642-7cbcced35c40 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.320584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae41a9-5669-438e-ac02-741d82fd328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.321253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae41a9-5669-438e-ac02-741d82fd328b None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.321533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae41a9-5669-438e-ac02-741d82fd328b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.321807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae41a9-5669-438e-ac02-741d82fd328b None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/60c809b5a47a4983be26116553934387/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.338448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae41a9-5669-438e-ac02-741d82fd328b None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=60c809b5a47a4983be26116553934387, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.339882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae41a9-5669-438e-ac02-741d82fd328b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.353735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 309/609] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:13 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/60c809b5a47a4983be26116553934387/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:13.718976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.846435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:13.916424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aadddb-f222-49a4-ab3d-2c4bc7eb3494 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.917214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aadddb-f222-49a4-ab3d-2c4bc7eb3494 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.917214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aadddb-f222-49a4-ab3d-2c4bc7eb3494 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.918597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 310/610] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:13.924723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.925048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.925371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.928088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb932a-495d-4179-a4be-d19719e983e7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.992266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.992542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:13.993204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.993703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.993972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:13.994267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.020484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:14.031846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:14.033723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:14.035070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 301/611] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:14.046719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.162165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 311/612] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:14.212960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7efb24-f3cc-4328-83d3-df63f95608eb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.248224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.258569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.259680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.259981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.260286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.261016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:14.261354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.262833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.273002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 302/613] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:14.281064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.388803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.441697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f315589-0f00-42e6-80e3-a8cb92383273 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.502725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.503301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.503574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.503840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.521147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.522681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:14.538565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:14.539694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:14.540720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:14.540720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:14.540720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:14.540720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 312/614] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:14.545631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.651722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.684521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875529-7654-4bdc-8443-4efd26d90225 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.737230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.737847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.738128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.738403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.739049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:14.739380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.741936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.765609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 303/615] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:14.771403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.830848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.860652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8bd7e-cc9f-4ca1-9bdd-a98aef1b4a01 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.887577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.923375 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.925483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.925763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.926037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.938095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b599c5-60b8-4da3-9c9f-18bf956d905b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.950270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:14.961354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:14.962333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:14.963168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:14.964360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 313/616] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:14.970730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.009446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.010099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.010375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.010640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.024704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.026074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:15.042816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.045366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:15.047881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 304/617] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:15.052029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.149306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437cfcc-ceb0-41b7-bdea-747a142dc5db None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.188818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce4a0-0925-4a97-96fa-c51479a02c66 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.194333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.195048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.195451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.195821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.197083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.197534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.199079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.201676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56ca78-2bde-4aa1-ad9d-d89bfe020b9b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.213802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 314/618] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:15.215912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a465e-a805-4641-96ab-1cad8ab320ab None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.216328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a465e-a805-4641-96ab-1cad8ab320ab None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.217233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a465e-a805-4641-96ab-1cad8ab320ab None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.219453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 315/619] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:15.220823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a465e-a805-4641-96ab-1cad8ab320ab None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.227605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.228397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.228560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.228732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.229460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.229767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.231219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb7775-a91b-46ee-9798-66baf25cd77e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.250774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 305/620] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:15.252156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.252514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.252792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.272056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:15.272675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.273264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:15.347452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 306/621] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:15.348689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.357737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a465e-a805-4641-96ab-1cad8ab320ab None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.403935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.404598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.404872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.405143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.416951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.418735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:15.434332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.435917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:15.436815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:15.436815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.436815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:15.439216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 316/622] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:15.444467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.457393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.464785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b840d4-6f8b-4c1c-ac5e-ce71f6d42237 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.515599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.516528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.516805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.517217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.528797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.530588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:15.548711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.549744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:15.551266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:15.551266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.551266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:15.551266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 307/623] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:15.556110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.575629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.594838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a99df-b895-47cb-a207-efaddc04252e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.658973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.659635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.659904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.660174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.660811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.661111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.662546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.676918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192af081-cf0c-4b60-8ef8-db1b3182e423 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.688021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 317/624] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:15.692998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.718436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.719426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.719932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.720407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.721272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.721782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.723380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.737340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 308/625] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:15.744206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.756405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.839496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d7e6e-b7d1-48ac-ac8d-45e4123ce76b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.865391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab75f6-ba0e-4a7d-80e6-57d107bbc3dc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.878772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.879565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.879842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.880114 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.889456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.890906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:15.904937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.905973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 318/626] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:15.907672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.910368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.917885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.920092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.920616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.921098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.933808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:15.940791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.941718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:15.942540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:15.944637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 309/627] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:15.946123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.023432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691991a-edec-41ef-b3fb-ba9838b9af09 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.028246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.061124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.061973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.062197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.062411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.068358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1caa80-835d-46ba-ad8d-79ccd50a2412 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.069266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.070726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:16.082554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:16.083542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:16.084412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 319/628] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:16.084899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.114484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.115155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.115473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.115823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.116565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.116888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.118309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a2bb6-558b-4b63-b219-86eca287c619 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.141733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 310/629] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.143340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95c126-50ba-44f9-8ebf-9753c5c2843e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.143683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95c126-50ba-44f9-8ebf-9753c5c2843e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.144020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95c126-50ba-44f9-8ebf-9753c5c2843e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.149388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 311/630] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:16.149388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95c126-50ba-44f9-8ebf-9753c5c2843e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.162877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.206899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6d3a7-d35b-4303-b6af-2b77ddbb949f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.244781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.245399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.245853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.245976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.246605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.246914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.248417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.255731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95c126-50ba-44f9-8ebf-9753c5c2843e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.261681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 320/631] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.261681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.291138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.303237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.304296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.304656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.306111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.307060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.307510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.308986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8b7ee-8e2e-464d-8aa2-ff7e121f6415 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.326985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 312/632] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.326985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.326985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.326985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.344008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:16.345289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:16.381784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78807698-0dda-43be-a370-44bb9a45c330 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.407314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.417804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 313/633] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:16.418095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd85d7-6160-4f59-8152-fcb2b72bff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.418584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.418729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd85d7-6160-4f59-8152-fcb2b72bff66 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.418964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd85d7-6160-4f59-8152-fcb2b72bff66 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.419260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd85d7-6160-4f59-8152-fcb2b72bff66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.499239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd85d7-6160-4f59-8152-fcb2b72bff66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.500973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd85d7-6160-4f59-8152-fcb2b72bff66 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.523824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f016e-69aa-4a8a-b9b0-bb2fa798d66d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.541344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.559899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.560573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.560857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.561132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.571504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.572992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:16.588314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:16.589288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:16.590162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:16.590162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:16.590162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:16.590162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 314/634] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:16.590162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.612518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 321/635] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:16.614071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d8fa7-d36d-4694-ab62-a1e072cef5cb None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.614524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d8fa7-d36d-4694-ab62-a1e072cef5cb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.614936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d8fa7-d36d-4694-ab62-a1e072cef5cb None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.616621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 322/636] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:16.617445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d8fa7-d36d-4694-ab62-a1e072cef5cb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.648263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.718128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972e75d-eeb3-4098-821c-f2605f7aa6a5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.735058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d8fa7-d36d-4694-ab62-a1e072cef5cb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.756593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.757252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.757528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.757801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/543542d17db0495f8151e779e6c6218f/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.771719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.772331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.772612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.772884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.773529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.773829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.774277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=543542d17db0495f8151e779e6c6218f, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.775266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05a2e-599d-4c63-9c91-8a5ef8058dbc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.775766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.788885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 315/637] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:16 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/543542d17db0495f8151e779e6c6218f/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:16.793883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 323/638] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.794057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.794137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.794137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.794137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.814482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.823697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:16.825052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:16.913588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 324/639] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:16.915620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.932940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e849a-6c71-40f5-8dfe-f1a5ed9c097c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.941615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.986923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.987888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.988228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.988568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.002615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.004411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:17.021565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.022736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:17.023732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:17.023732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.023732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:17.023732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 316/640] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.023967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.056596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc50ea-665b-40f9-a2fc-46120d724d83 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.102381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.103407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.103712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.103989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.106298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.119014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:17.126853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.127855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:17.128753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:17.131215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 325/641] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.132216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.150803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258401ee-acd6-43c3-93a5-e80c2027a5e3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.195742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.197167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.197889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.198382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.209380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.212056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.226324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:17.227190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.228217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:17.229162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 317/642] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.229791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.259678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49b509-ced5-46eb-a290-98a23efc5993 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.297314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.297930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.298233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.298503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.299142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.299492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.300884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.313412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 326/643] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.314244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.347029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddbe5b-283c-4d5b-942d-2b3959714724 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.379932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.387962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.388604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.388880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.389149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.389805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.390102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.391589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.404908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 318/644] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.404908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.421657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182bb91-8a40-47d0-b69d-e6655db5e261 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.455447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.456101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.456381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.456690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.458601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.468915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:17.474251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.475138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 327/645] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.476187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.525291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d98847-e644-45dc-8adb-2a005dedcb06 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.561602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.562213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.562484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.562753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.563505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.563806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.565228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.583290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 319/646] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.583964 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.590757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44985f-0aa6-41b9-8136-ac93cbb85731 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.625110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.625741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.626018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.626288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.633127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.634849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.637091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:17.646999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.648100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 328/647] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.648985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.697218 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cc684-9309-411f-8b6e-68571d52b9c0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.707954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.737711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.738350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.738624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.738890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.749594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.751103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.756419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e488f-d422-4bf8-a6a3-aca2c6b3f7b1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:17.766093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.767024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:17.767950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:17.767950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.767950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:17.767950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 320/648] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.767950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.796106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.796708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.796983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.797271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.797897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.798197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.799598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.807722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 329/649] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.808795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.880116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e0b5-94f9-4eeb-a8c2-b0e2b9f9b49b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.882172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.918471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.919067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.919195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875b512-16c1-41f3-9aef-42c5fa045b43 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.919397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.920188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.936616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.956227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.956828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.957107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.957382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.958060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.958367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.959836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.972704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 330/650] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.973531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.998802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.000443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.081889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d93370-2491-4808-86a9-43d534f73bcb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.116019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 321/651] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:18.117012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.119741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.121059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.121682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.121932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.122213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.122881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.123194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.124605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.136911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 331/652] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.137767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.181029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.226946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96f47e-fb5a-4cf8-973e-fa4743aa3f86 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.252170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a6d10-e3b6-46a0-9a52-1a9f4eeb4987 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.261780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.262414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.262667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.262997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.275052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.276512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.288203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.288811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.289098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.289440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.290091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=ff1a6a40166e4f51a14cca9cd510e680) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.290403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:18.291752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.292909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:18.294152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:18.294152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:18.294152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:18.294320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.298506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 322/653] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.307673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.307864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 332/654] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/role_assignments?user.id=ff1a6a40166e4f51a14cca9cd510e680&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.307864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.370358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.435871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.437631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d6c6b-4efc-4fb4-989e-4d361defe8de None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.442136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1a110-7fa8-451e-a2a5-c7a0c4f24d6d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.498102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.498763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.499044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.499352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.506141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.507714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.508226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.509038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.511617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.513895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.520505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.522138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:18.531778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:18.534037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:18.534037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:18.534037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:18.534037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 323/655] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.534037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:18.538166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.539139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:18.540187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 333/656] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.543827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.662736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f187d0-cf40-4127-ae1b-8197dc47c988 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.664510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a97e0-f83b-48fd-b2ad-bf2532cd26b7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.670993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.706205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.707302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.707997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.708755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.709817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.710387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.711889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.712252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.712495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.712770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.713099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.713796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.714150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.715889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.726539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 324/657] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.731233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.733625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 334/658] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:18.739184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.769132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.862905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76175ba-a936-4dd6-98a4-f565dbaa3025 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.877946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe13d5-28b8-469f-b594-660c5470e658 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.914111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.915116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.915639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.916130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.916921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.917560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.917839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.918113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.918870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.919268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.921346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.928665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.930886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.942933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 325/659] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.943887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:18.951034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.952064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:18.952974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:18.956279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 335/660] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.957361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.977347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.055974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.070918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273c478-883d-48d4-9d11-12477caba589 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.079098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759374e4-f9c5-4b84-8958-cbe1346e41cd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.109376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.110269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.110552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.110823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.118954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a5446-71f3-40e1-91c8-4d477147f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.119638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a5446-71f3-40e1-91c8-4d477147f19b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.120070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a5446-71f3-40e1-91c8-4d477147f19b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.120523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a5446-71f3-40e1-91c8-4d477147f19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.121563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.123764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:19.138852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.144955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 326/661] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.145896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.205447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a5446-71f3-40e1-91c8-4d477147f19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.206913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a5446-71f3-40e1-91c8-4d477147f19b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.239801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.258520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321f544-511b-4c67-a65a-69291dd0b490 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.292449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.296453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.297318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.297628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.298142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.298695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.299001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.300472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.314484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 336/662] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:19.315922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab85651-1a1f-40c8-a302-bb23d682244c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.316104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab85651-1a1f-40c8-a302-bb23d682244c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.316323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab85651-1a1f-40c8-a302-bb23d682244c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.317537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 337/663] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:19.318225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab85651-1a1f-40c8-a302-bb23d682244c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.332832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 327/664] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.334630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.428557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab85651-1a1f-40c8-a302-bb23d682244c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.444664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385605e-ca44-4d4a-9858-d54a314c6162 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.465070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.465874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.466837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.467293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.468251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.468728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.470385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa64f-6af3-49cb-a8f2-a73860d84f3f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.486286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.487027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.487429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.487747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.488693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 338/665] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.490131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.490422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.490702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.499585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.501021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.502825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.510057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:19.511300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:19.519962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.520891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:19.521800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:19.521800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.521800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:19.523506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 328/666] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.524750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.548097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.584195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 339/667] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:19.585112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.648947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952bbb5-e4b0-46dd-9c8c-f374ae58ca01 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.686760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.687408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.687718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.687959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.713795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23271-51f8-4af8-bfaa-6fdc8021ffca None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.761426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.761634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.762678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.763728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.763728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.765513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=623f9046c3924b35b36d8807f8eef54c) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.765829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.767500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.775888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.777465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.777686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 340/668] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/role_assignments?user.id=623f9046c3924b35b36d8807f8eef54c&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.778959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.830256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.907755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 329/669] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:19.914068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.921115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24096cd1-283c-40d9-a497-c4b50512d21c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.967944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.968993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.969485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.970342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.971237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.972084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.974003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.992505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 341/670] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.993893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.029398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.040009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2d412-1bac-4c01-9cf0-36413a3d6948 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.088423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.088632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.089001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.089280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.105579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:20.106434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:20.113086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.114441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 330/671] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.115347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.120293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b909bfc-7412-45f8-801d-4f37003e964f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.160068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.160957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.161408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.161932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.173702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.175604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:20.192270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.193289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:20.194251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:20.194251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:20.194251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:20.196379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 342/672] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.202044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.240975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5eb9e1-3d60-436a-acf7-8d3fd8d6aac6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.284653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.285671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.286025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.286366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.295404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.296958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.310471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554f553-26dd-45d8-82bb-5c4db3783c7f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:20.312723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.313770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:20.314907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:20.320306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 331/673] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.320306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.337555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.365513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.366130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.366409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.366681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.367380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.367748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.369140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.381227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 343/674] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.382968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.399904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.450083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23475e42-b5a1-448b-8805-d0bca0899438 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.509638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.510529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.510951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.511386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.511785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff842b16-b793-4d20-bea2-7540a12fbd97 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.512336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.512777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.515515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.533946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 332/675] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.535101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.553779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.555124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.555693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.556209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.557055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.557756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.559769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.577891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 344/676] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.578992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.631962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.668977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f904-6635-4897-8afe-1c2c6e485eff None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.671144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.711501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cdeee-6c35-42fd-9443-15c4311ebc52 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.718347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.719917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.719917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.719917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.733608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.735279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:20.754543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.756807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:20.758131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:20.758131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:20.758131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:20.758131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 333/677] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.758131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.768836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.769609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.769983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.770671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.778033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.779789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-688a3fce-1172-4b9c-8f3b-5cc57c10da6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:20.792050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.793242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:20.794506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:20.794978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 345/678] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.795956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330a62b-9837-4c76-a98e-4ece185d7885 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.796240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330a62b-9837-4c76-a98e-4ece185d7885 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.796505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330a62b-9837-4c76-a98e-4ece185d7885 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.801948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 346/679] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:20.801948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330a62b-9837-4c76-a98e-4ece185d7885 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.872085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858a45b-2c04-42d7-b98f-68c3d680a6c5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.905974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.917007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.917712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.918082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.918425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.919265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=b4e09c957a104f2299ab55d8283124c5) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.919586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.921067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330a62b-9837-4c76-a98e-4ece185d7885 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.921202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.927648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 334/680] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/role_assignments?user.id=b4e09c957a104f2299ab55d8283124c5&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.928475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.962713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.976263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.977280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.977647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.978025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.979956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.980271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.982426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb85aa-2b02-4344-b096-971e62c6fd37 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.999986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 347/681] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:21.001299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.001588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.001862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.030213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:21.032848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:21.077302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e387cf-9a5f-4488-a04b-90be0bc092d8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.115259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 348/682] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:21.116199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.127814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.128445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.129236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.129513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.130682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.131005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.134289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.150140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 335/683] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:21.156244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.223013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.242493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca051e-5576-4517-80c9-cc911af9228a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.262938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.285410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.286047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.286322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.286590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.295304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c3229-0281-4f24-a9cb-9b508bca0de8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.301192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.302659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:21.316974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:21.317977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:21.318950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 349/684] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:21.324115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.341587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.342592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.343098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.344126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.345420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=60c809b5a47a4983be26116553934387) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.345926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.347913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.355527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 336/685] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/role_assignments?user.id=60c809b5a47a4983be26116553934387&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:21.432598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.467220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7418d24-31dc-4fdd-926c-60429561acaa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.569192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.569820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.570099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.570376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.571617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.571929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.573766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.605234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 350/686] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:21.611429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.614620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0d07-87de-4c61-a26e-f2f5901b351f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.635469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.657116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.670743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.671439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.671728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.672011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.685485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:21.691895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:21.692871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:21.693765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:21.695283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 337/687] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:21.699065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.757334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b727e-d2cc-403b-ae05-33dcafe702c7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.796592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.797217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.797493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.797764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.832127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331e12-5233-4edb-b8a8-5a44b5253ce5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.886880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.889567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.903278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.903937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.904210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.904480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.905146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.905446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.906815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.926208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 338/688] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:21.932092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.965050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.024773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 351/689] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:45:22.030570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.037754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.110607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627bbd2-35db-4a0c-a625-5926e35ab88e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.149606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.150531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.150965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.151425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.165528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.167816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:22.184699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:22.186447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:22.187594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:22.187594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:22.187594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:22.189402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 339/690] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:22.196300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.219229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1bc2a-60ba-4ff0-b586-16cf2fce45f6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.284379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.285196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.285482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.285759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.289795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.307897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.309364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2b75557d-15a9-4ace-ab40-cc0c2a3f421e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:22.326287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:22.327295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:22.328260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:22.328260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:22.328260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:22.328260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 352/691] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:22.329043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff377e-986d-47eb-bd0e-4926b457a984 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.329330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff377e-986d-47eb-bd0e-4926b457a984 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.329604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff377e-986d-47eb-bd0e-4926b457a984 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.331238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 353/692] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:22.332809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff377e-986d-47eb-bd0e-4926b457a984 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.355542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.372435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899578b-9d28-43c6-83c2-5196a9a1a0dd None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.416171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.416700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.416979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.417250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.417892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:22.418193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.420892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868391b4-802a-47bd-ae70-5097957a50b9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.439292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 340/693] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:22.439292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.439292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.439292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.453517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff377e-986d-47eb-bd0e-4926b457a984 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.463312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:22.464576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:22.499860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.500501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.500780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.501056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.501701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:22.502006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.503529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.517473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 354/694] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:22.518428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.571325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 341/695] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:22.571325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.584866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.677099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e85de-9ffe-4179-96ae-1ae31faac014 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.719378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.745774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.746408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.746686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.746958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.764343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.765856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.771322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1c728-445e-49f0-91a0-4959acd5a172 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:22.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:22.792695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:22.793920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:22.806936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 355/696] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:22.812298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.839654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.840533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.840964 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.841959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.854547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.856211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:22.873232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:22.874374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:22.875540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:22.880753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 342/697] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:22.886498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.967002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da2e22-5f29-49e6-a118-28787954d98e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.976990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.006999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.007819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.008165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.008503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.009632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.010001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.011575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.016133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368308e2-9c21-4747-8a04-f0c2a1d11253 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.036285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 356/698] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:23.037478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.053821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.054837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.055333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.055843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.056803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.057313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.059160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.067634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.073995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 343/699] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:23.075132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.162558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2252c93-4b14-40e9-990b-5e23e0b6faf0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.190464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca34f-4473-4a80-8b69-2e96663fa44c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.201945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.202953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.203433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.203844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.215240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.216965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.229076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.229749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.230129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.230509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:23.234641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:23.235631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:23.236509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:23.236509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:23.236509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:23.238040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 357/700] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:23.239019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.261296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.303748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.309194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.311537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.354643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028bf6f-37d3-426c-82f7-d01c11308a27 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.393579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.394490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.395104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.395638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.396666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=543542d17db0495f8151e779e6c6218f) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.397165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.398782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.406034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 358/701] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/role_assignments?user.id=543542d17db0495f8151e779e6c6218f&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:23.407239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.423287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 344/702] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:23.427800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.526617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.547581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562aec-ab3c-4f06-a202-2ed3494449a0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.570318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.582993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a86fb4-8d17-4c9b-8f33-26672e012596 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.597257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.598282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.598794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.599374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.602469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.603109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.604802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.633921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 359/703] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:23.641144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.654240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.654882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.655160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.655461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.666079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.667559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:23.694958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:23.696180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:23.697061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:23.697061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:23.697061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:23.697061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 345/704] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:23.702999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.789140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9563bb-00d5-491a-aabd-895d8fe197f4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.840318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704355f-cca5-4e8e-8e43-dbe78095b972 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.840861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec5d2e-989c-466a-bcb5-a75d52c8737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.841820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec5d2e-989c-466a-bcb5-a75d52c8737d None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.842097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec5d2e-989c-466a-bcb5-a75d52c8737d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.842555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec5d2e-989c-466a-bcb5-a75d52c8737d None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.843079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec5d2e-989c-466a-bcb5-a75d52c8737d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.845065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec5d2e-989c-466a-bcb5-a75d52c8737d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.863880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.871256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 360/705] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:45:23 2022] POST /identity/v3/users => generated 254 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:23.873011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119699b-4705-4c47-95ec-417b3145f232 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.873356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119699b-4705-4c47-95ec-417b3145f232 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.873633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119699b-4705-4c47-95ec-417b3145f232 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.878965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 361/706] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:23.881873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f6b6-69f6-4bed-9558-38af2751b4d8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.882220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f6b6-69f6-4bed-9558-38af2751b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.882506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f6b6-69f6-4bed-9558-38af2751b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.897988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.898626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.898910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.899188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.904851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.905067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3ce3f6b6-69f6-4bed-9558-38af2751b4d8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:23.905152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.906318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3ce3f6b6-69f6-4bed-9558-38af2751b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:23.906578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.937004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 346/707] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:23.945559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.968407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.996765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 362/708] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:24.123869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81836637-dd1b-47d1-9c2e-3cd918a831c6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.124208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81836637-dd1b-47d1-9c2e-3cd918a831c6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.124464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81836637-dd1b-47d1-9c2e-3cd918a831c6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.127227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 363/709] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:24.135998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735430ea-a67c-4119-a926-8282dc050277 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.136314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735430ea-a67c-4119-a926-8282dc050277 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.136551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735430ea-a67c-4119-a926-8282dc050277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.139664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79decd-89bc-447e-9857-4dc0c190d429 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.162505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-735430ea-a67c-4119-a926-8282dc050277 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:24.164668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-735430ea-a67c-4119-a926-8282dc050277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:24.220254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.220903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.221258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.221532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.231289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.233759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:24.249683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:24.250634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:24.251543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 347/710] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:24.256872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.257159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 364/711] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:24.313943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735430ea-a67c-4119-a926-8282dc050277 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.333492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.371516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a995d5-3365-4a6d-bfb8-010fd60b8c34 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.383990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.414068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.414680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.414956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.415939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.415939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.416209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.417843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.432169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 348/712] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:24.438868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.444226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735430ea-a67c-4119-a926-8282dc050277 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.494154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.495413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.495925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.496438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.497280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.499095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:24.517551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:24.518664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:24.519655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:24.529919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 365/713] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:24.536205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.557420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fb5cf-a0a1-4441-9b5f-05093414bf60 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.587770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.594312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.595079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.595240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.595682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=ff1a6a40166e4f51a14cca9cd510e680) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.597696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.600078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.605742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 349/714] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/role_assignments?user.id=ff1a6a40166e4f51a14cca9cd510e680&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.607342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.660274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d16529-fd3c-4d15-abdc-08bc0005d41a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.695682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.700170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.701104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.701600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.702110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.702940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.703491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.705185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.710632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 366/715] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.717321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.768143 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7d3e9-6f6f-4472-8a2c-8feb30c78e29 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.832237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619ed19-401b-4c51-b898-5d4c379f7ca8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.833382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.834037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.834318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.834594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] PATH_INFO: `/v3/services/network` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.835183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.836919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:24.866335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:24.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:24.868719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:24.872001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 350/716] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:24.875778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.876689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.877168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.877640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.878498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.879772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.883164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.885991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.898766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 367/717] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.972270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:25.030230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36794c18-45ed-4e08-968c-ff246eb27241 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:25.074989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:25.076188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:25.076463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:25.076737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:25.077377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:25.077677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:25.079085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e706a-75a3-4584-ad38-78fa805278d1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:25.084768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 351/718] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:25.090278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:25.151363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:25.208213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52096406-d4d1-4d28-a029-3bae5fc62136 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:25.242738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:25.243310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:25.243592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:25.243923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:25.244558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] RBAC: Adding query filter params (type=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:25.244921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:25.246344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80138d-8d11-4086-9f46-dd68842effee None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:25.252061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 368/719] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:26.114227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99ef9f-10ad-4df5-b2db-ac40f4094f89 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.115122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99ef9f-10ad-4df5-b2db-ac40f4094f89 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.115607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99ef9f-10ad-4df5-b2db-ac40f4094f89 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.117083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 352/720] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:26.123315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.124077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.124518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.152193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:26.154153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:26.237629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 369/721] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:26.570695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99ef9f-10ad-4df5-b2db-ac40f4094f89 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:26.704908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99ef9f-10ad-4df5-b2db-ac40f4094f89 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:26.732685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:26.750973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:26.751631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.751910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.752182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.768479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:26.773414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:26.774343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:26.775238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 353/722] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:26.780529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:26.847368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:26.939190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31405c-c97d-4327-a3e5-70003c178521 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:26.988156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:26.988689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.988967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.989239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.989904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:26.990208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:26.991609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76ebf7-d229-4ce4-b7c0-1a441e3ee312 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.000790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 370/723] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:27.006602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.126409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33219402-3769-401b-b481-a5afd4bc59fd None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.161592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.162233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.162511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.162784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.174205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.175703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.184081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af0921-0c7d-47b3-b363-01fff1b4a06f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.185315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af0921-0c7d-47b3-b363-01fff1b4a06f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.185315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af0921-0c7d-47b3-b363-01fff1b4a06f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.186875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 371/724] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:27.191841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98461cc8-bed2-4476-bea3-b1f0f4ee3ef6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.192122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98461cc8-bed2-4476-bea3-b1f0f4ee3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.192387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98461cc8-bed2-4476-bea3-b1f0f4ee3ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:27.196211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:27.198355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:27.200004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:27.200004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:27.200004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:27.200004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 354/725] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:27.207838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.227388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-98461cc8-bed2-4476-bea3-b1f0f4ee3ef6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:27.230571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-98461cc8-bed2-4476-bea3-b1f0f4ee3ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:27.266698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.340058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04034d-11e6-4c66-b6de-c122d8d52ae8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.369083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 372/726] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:27.370459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8406a3-da40-4ae6-ac7a-be25a16d150b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.370745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8406a3-da40-4ae6-ac7a-be25a16d150b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.371015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8406a3-da40-4ae6-ac7a-be25a16d150b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.372889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 373/727] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:27.379429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81936db7-7978-4f28-8cf8-858d984237ff None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.379429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81936db7-7978-4f28-8cf8-858d984237ff None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.379429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81936db7-7978-4f28-8cf8-858d984237ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.386704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.387357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.387646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.387999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.388621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:27.388928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.390289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.409627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 355/728] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:27.410904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-81936db7-7978-4f28-8cf8-858d984237ff None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:27.412135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.413778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-81936db7-7978-4f28-8cf8-858d984237ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:27.512953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.560274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.577389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa0bb-7db5-442a-96fd-7d3ee709daa6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.598952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 374/729] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:27.600559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca931-522e-4649-9ced-563b1f2ce8f1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.600845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca931-522e-4649-9ced-563b1f2ce8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.601120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ca931-522e-4649-9ced-563b1f2ce8f1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.603340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 375/730] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:27.611466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.612046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.612523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.625494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.626127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.626403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.626672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.636662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.638213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.639340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:27.641688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:27.651016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:27.652168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:27.654139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 356/731] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:27.658634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.737077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 376/732] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:27.738261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.843705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.851844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34bacb-5e33-4344-84b1-c7898bb6b227 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.857403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60abba4-00d4-4b70-9df9-68eaad2ca8b8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.906537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.907530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.907996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.908536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.920972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:27.927235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:27.928172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:27.929000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:27.931361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 377/733] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:27.939340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.954852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.955613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.955893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.956162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.957035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:27.957342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.958870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.979476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 357/734] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:27.980427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.996302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.065960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6b3ed-1852-4a98-b777-5577b75d753b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.101000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370697d-c7cc-4276-b04e-d5af86812468 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.102754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.103562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.103838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.104110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.104767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.105086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.106463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.115520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 378/735] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.116386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.137190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.144910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.145857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.146146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.146424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.159451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:28.164531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:28.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:28.166491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 358/736] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:28.167179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.235506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6333-e44d-42a3-98aa-f44aba0f7b20 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.243567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.273902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f01ecf-9717-494c-af70-30b3912373f2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.274098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.274698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.275000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.275401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.310109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.310716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.310991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.311304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.311949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.313315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.329358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 359/737] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:28 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:28.330269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.348930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.350420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.371522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.472286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 379/738] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:28.472286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.478945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7ae02-cdbb-4cd8-aab8-7fad19765eb5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.513193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.523535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.524167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.524452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.524907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.538520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.540015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b91ecd1a-320a-4db5-8d63-3a5fd0233ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:28.556663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:28.557663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:28.558821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:28.558821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.558821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:28.558821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 360/739] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:28.559851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b362-c1e9-48c8-9a79-7b533aed62c6 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.559960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b362-c1e9-48c8-9a79-7b533aed62c6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.560195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b362-c1e9-48c8-9a79-7b533aed62c6 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.561577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 361/740] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:28.562122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879b362-c1e9-48c8-9a79-7b533aed62c6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.591598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e19760-a249-46cf-b513-d782b5c76397 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.610019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.629155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.629729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.629957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.630223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.630880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.631181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.632575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.642063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 380/741] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.647136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.678255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879b362-c1e9-48c8-9a79-7b533aed62c6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.714073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.714734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.715013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.715384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.728945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.730397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-447bc07e-1a05-4293-be06-f71f71eed6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:28.745461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:28.746520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:28.747514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:28.747514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.747514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:28.751382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 362/742] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:28.752621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.752909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.753188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.754247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d58715-d6b5-4a96-8d1c-4e0cf9edb342 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.772088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:28.773293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:28.789785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.790393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.790671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.790959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.791615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.791922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.793286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.802108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.811891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 381/743] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.811891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.852999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 363/744] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:28.853853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.879341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.953766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598df5-d3dc-4ae9-b55a-fb3fbe77f088 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.977869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b0c86-fcd0-4a71-af94-3582e9e927b2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.998037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.999010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.000026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.000526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.014868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.015741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.016024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.016297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.016661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.016937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.017238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.018605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.018905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.036773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 364/745] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.037660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:29.038074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.038973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:29.039877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:29.039877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:29.039877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:29.043167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 382/746] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.044363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.067749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.160255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.161107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db3eff-e41b-4459-86b4-b22a01398e15 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.170539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e370302-2041-4bbe-97b5-90da6c0b214a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.198966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.199828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.200235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.201083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.205152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.205784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.206062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.206335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.210757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.213474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.213697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.214909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:29.227845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.229228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 383/747] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.230782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:29.232111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.233145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 365/748] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.240529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.338415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.342707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d89ee-737d-4d90-8fcd-91fc648e8bd9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.362256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6696ad1-9209-4c8b-84cb-697a48d705af None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.377872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.378478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.378752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.379031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.379722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.379984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.381367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.399376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 384/749] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.400485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.402784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.403573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.403849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.404222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.404879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.406263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.415327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.427255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 366/750] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:29 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:29.427255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.532626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6a30-2d24-47aa-b487-232b42aa009f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.581154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.581766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.582041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.582447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.583155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.583823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.585109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.592392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb7bda-cd9b-4014-a73d-51b754610f5d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.607078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 385/751] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.608097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.632853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.633465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.633994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.634263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.634958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.635295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.636747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.652494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 367/752] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.654947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.657986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.692973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.744775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f3dc-5c22-4e35-8d8f-058e58ec27bc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.778924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f075d-c23e-4f7e-a6a0-53505f763108 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.787694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.788497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.788845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.789198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.799624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.802190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:29.816857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.817851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:29.818786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:29.819358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 386/753] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.820585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.822202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.823045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.823638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.824239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.899866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.901508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.904657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.940532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752be99-645f-4305-b665-7068e61a3d9f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.969131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.979173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.980643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.981316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.982566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.983758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=543542d17db0495f8151e779e6c6218f) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.984254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.985932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.992688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 387/754] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/role_assignments?user.id=543542d17db0495f8151e779e6c6218f&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.993828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.037190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 368/755] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:30.044434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.119764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73eb73-a3a1-4622-9757-15ed05d81ef4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.166178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.167270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.167788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.168376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.169492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.170025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.171741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.193902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 388/756] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:30.199984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.211505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9177a54-da33-4a4b-9124-bc762853a924 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.213243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.261063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.262135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.263188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.263715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.275812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.276122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.277800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:30.295910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.297112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:30.298034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:30.298034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:30.298034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:30.299576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 369/757] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:30.304186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.335424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190c48-4725-4c60-b3ea-2da1edfd9c92 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.372937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.373568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.373785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.374014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.385144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.386749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:30.402156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.403159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:30.405418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:30.405418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:30.405418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:30.405418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 389/758] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:30.409660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.533405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6a01a-0330-4933-bbfd-202f3f2d2628 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.552993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.561264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd1d41-ca1c-420f-9b68-614897f5145c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.581962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.582585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.582867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.583144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.583921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.584234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.585798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.599292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 370/759] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:30.604773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.623049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.626417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.627040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.627354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.627627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.628275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.628582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.629953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16063e5-4b05-426c-98b9-d2504e7c32a2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.652310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 390/760] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:30.658893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb79b93-5f92-40a6-8f99-6ab7a3baffea None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.659488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb79b93-5f92-40a6-8f99-6ab7a3baffea None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.660000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb79b93-5f92-40a6-8f99-6ab7a3baffea None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.661748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 391/761] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:30.662665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb79b93-5f92-40a6-8f99-6ab7a3baffea None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.737710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f3f9-2e5c-4980-a14c-5d788c5ba1f6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.778240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.778870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.779151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.779596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.786741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.788201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.794453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb79b93-5f92-40a6-8f99-6ab7a3baffea None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c6a629e2-7f90-49ff-b74b-45cda0c1bdc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:30.800848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.801760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:30.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:30.806748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 371/762] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:30.808074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.808358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.808704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.828641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:30.829844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:30.848390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.850338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.850861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.851401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.862860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.865105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:30.881243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.882196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:30.883071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:30.889534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 392/763] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:30.889534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.916093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 372/764] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:30.917256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.921248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.925473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.042249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091771f-9144-4e8d-80ab-153d4f37eeb1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.087818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.088433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.088707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.088971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.089658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:31.089981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.094019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.096755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf3968-b652-472a-8f3f-68a34f1e2cc7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.118523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 373/765] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:31.126067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.162683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.163355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.163631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.163898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.164601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:31.164906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.167605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.196516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 393/766] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:31.205033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.243016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.299435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b40c8-3102-4588-bea4-3d1b95d933f3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.356963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.360266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.361129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.361550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.362020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/ff1a6a40166e4f51a14cca9cd510e680/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.364883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=ff1a6a40166e4f51a14cca9cd510e680, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.366638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.379819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e77e4-c0fd-4522-aacc-052e162ba206 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.385545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 374/767] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:31 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/ff1a6a40166e4f51a14cca9cd510e680/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:45:31.386642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.464653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.466864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.468670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.469207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.471795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=1e8df64cf5134a7e957de60145c8783b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:31.472402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.474913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.492821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 394/768] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/role_assignments?user.id=1e8df64cf5134a7e957de60145c8783b&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:31.573769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9301c5-5e31-4ded-b5b0-84b90f04afea None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.614472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.615130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.615438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.615708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.616295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.617677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.619762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:31.627901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:31.628968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:31.630904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 375/769] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:31.636165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.758097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302f814-e4f5-460d-87c3-c746f808324d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.794320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.794932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.795227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.795522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.796148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:31.796446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.797871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 395/770] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:31.808377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.975323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33b949-ee65-4964-a38f-d4b8820dbe0a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.026283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.026897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.027177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.027475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.028622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.028937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.030342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e832b-c496-4d42-a666-54aa66d7dbb1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.040602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 376/771] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.046971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.068520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.172534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fbb94-eaf7-4e0e-8aff-20be76a373da None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.172843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fbb94-eaf7-4e0e-8aff-20be76a373da None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.173095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fbb94-eaf7-4e0e-8aff-20be76a373da None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.174647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 377/772] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:32.180230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f51a-1a9a-4e3e-9038-4d3eec45b00b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.180512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f51a-1a9a-4e3e-9038-4d3eec45b00b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.180791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7f51a-1a9a-4e3e-9038-4d3eec45b00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.198296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d58c0-6f43-4454-979d-a024505ad8a3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.210882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0ce7f51a-1a9a-4e3e-9038-4d3eec45b00b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:32.212124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0ce7f51a-1a9a-4e3e-9038-4d3eec45b00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:32.267907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.268444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.268724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.269000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.269710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] RBAC: Adding query filter params (interface=public, service_id=3ee175417a3c47b6969266ae7ed74a76, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.270020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.271419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.284245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 396/773] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/endpoints?service_id=3ee175417a3c47b6969266ae7ed74a76&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.289726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.331179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.364009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 378/774] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:32.408041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfa816-bd37-42a0-9263-d5f1ffaf9987 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.411534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.446069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49391-e965-4516-8d02-0cd970c4d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.446739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49391-e965-4516-8d02-0cd970c4d32e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.447027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49391-e965-4516-8d02-0cd970c4d32e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.447332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49391-e965-4516-8d02-0cd970c4d32e None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.447947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d49391-e965-4516-8d02-0cd970c4d32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.449333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d49391-e965-4516-8d02-0cd970c4d32e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.455744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 397/775] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:32.545782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bf59-2bf4-45c6-a6e9-748de0e6f621 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.546125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bf59-2bf4-45c6-a6e9-748de0e6f621 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.546332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bf59-2bf4-45c6-a6e9-748de0e6f621 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.548030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 379/776] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:32.553542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.553852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.554101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.590576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:32.592487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:32.672101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 398/777] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:32.690972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370bf59-2bf4-45c6-a6e9-748de0e6f621 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.796812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370bf59-2bf4-45c6-a6e9-748de0e6f621 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.831615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.832620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.833105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.833578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] PATH_INFO: `/v3/services/network` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.834362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.835968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:32.847233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:32.848323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:32.849972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:32.857300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 380/778] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:32.858918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.935599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.986901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656314ae-7b5b-40ba-b987-bd220c9a0f6e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.002328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.062296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.063262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.063843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.064305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.065138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] RBAC: Adding query filter params (name=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:33.065635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.067230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.076209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 399/779] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:33.084548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.084706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.201696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3b24b-23e0-4966-bf86-97224427e891 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.258800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99133c94-6547-4332-8f9c-d6c57cc410cf None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.274752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.275265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.275546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.275825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.291140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.300458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:33.304664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:33.305554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:33.306767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 381/780] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:33.311987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.312556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.313148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.313619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.314093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.315015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] RBAC: Adding query filter params (type=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:33.315553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.317242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.331552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 400/781] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:33.336816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.450040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1beb17-73a7-46c1-b991-02c7a1882a6a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.450661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c76aab-aa2b-4c45-8ffe-bd431e0b4df3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.462661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.502897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.503538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.503819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.504103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.504746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:33.505045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.506419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.522507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 382/782] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:33.528032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.533097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.534016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.534506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.534982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.535921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] RBAC: Adding query filter params (interface=public, service_id=8add776ce28241ca9aeab6719f5f59f7, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:33.536426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.538104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.554312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 401/783] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/endpoints?service_id=8add776ce28241ca9aeab6719f5f59f7&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:33.559540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.631788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2d8e9-e6e0-48d1-ba4e-fb24dd120f4b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.666698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.667519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.667977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.675289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.675289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.680711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.682297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:33.697004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:33.698144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:33.699238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:33.699238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:33.699238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:33.701181 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 383/784] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:33.709143 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.729408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.746741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b66c47-ef5b-4ad6-a887-b0a1709e3a99 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.789974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.790862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.791297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.791692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.792417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.794014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.808339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 402/785] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/services => generated 772 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:33.861470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03a06-dbbb-4c34-955c-23b1f4a5b59d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.918591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.919597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.920021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.920466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.921276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:33.921731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.923312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dc3df-546d-4520-9e82-cb65d1bb03be None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.941634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 384/786] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:33.947732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.060285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6fe8e-90e7-4b01-ad46-7c0a1d431a39 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.060958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6fe8e-90e7-4b01-ad46-7c0a1d431a39 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.061441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6fe8e-90e7-4b01-ad46-7c0a1d431a39 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.063466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 385/787] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:34.069692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4fe4d-c443-4688-ae81-8ca9468d3132 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.070668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4fe4d-c443-4688-ae81-8ca9468d3132 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.071153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4fe4d-c443-4688-ae81-8ca9468d3132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.096630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5ce4fe4d-c443-4688-ae81-8ca9468d3132 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:34.098222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5ce4fe4d-c443-4688-ae81-8ca9468d3132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:34.122875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178083d3-b19f-456d-8df5-3b5134aa7619 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.168033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.186422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.186422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.186422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.186422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.196205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.198180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.208375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 386/788] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:34.209775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36383638-0bb4-4513-9c2e-d27d7824bbe1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.210066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36383638-0bb4-4513-9c2e-d27d7824bbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.210344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36383638-0bb4-4513-9c2e-d27d7824bbe1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.211942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 387/789] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:34.215729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.216658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:34.217607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:34.218132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.218132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.218132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.220347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 403/790] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:34.228321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.236669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:34.237867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:34.242482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.329103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 388/791] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:34.386216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8942d-167a-45b4-bbdf-579aaa91d287 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.422823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.423850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.424352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.424827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.425753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:34.426264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.427957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.452901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 404/792] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:34.458575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.526830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.608207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.624249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2291e-f7dc-4719-9d42-ac27df2e191b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.676797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.677438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.677711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.677982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.690133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.691751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:34.715532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.716493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:34.717371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:34.717371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:34.717371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:34.717371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 389/793] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:34.717371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.719965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.728017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbda6b-9381-4732-bcd3-9299ce8bfad5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.765944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.766930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.767380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.767792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.779448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:34.784859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.785961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:34.786971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:34.789333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 405/794] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:34.790267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.830220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6a26-22cf-4277-b7bc-1dc418018016 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.867785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.868533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.868813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.869084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.881392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:34.887643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.888625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:34.889480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 390/795] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:34.890189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.910276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.915288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992b1d-9138-4201-b600-742a997afccc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.953791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.954341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.954925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.955466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.955919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.956739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:34.957243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.958819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.977551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 406/796] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:34.978744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.002955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f5a42-8e60-4f83-b1b4-5cd0f3f92c56 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.044291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.044916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.045195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.045469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.046210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.046434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.047848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.062583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 391/797] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.062583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.097253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70248fe4-fee9-4b19-b601-e98db7571731 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.139378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.139994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.140272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.140545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.141207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.141510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.142889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.152191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 407/798] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.153674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.179569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d4e2e-95c1-4afa-b9a3-5aa2a7d1aa9d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.187608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.195307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.226303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.227892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.228424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.228847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.239458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.241643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:35.257420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:35.258793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:35.259916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:35.262087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 392/799] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:35.263593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.281020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42be03e-031c-4693-8a61-5ac4b99ff279 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.329528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.330173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.330451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.330723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.342625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.344143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:35.359873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:35.360860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:35.361732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:35.361732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.361732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:35.361732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 408/800] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:35.361962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.405344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b2f9b-5344-4043-a145-d1e67fa664b4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.445854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.446757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.447437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.447725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.447998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.460023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.461495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.483634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cfc33-bf61-4cc3-b106-ccb1796508c4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:35.490561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:35.491697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:35.497441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:35.497441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.497441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:35.497441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 393/801] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:35.497441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.525226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.527728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.529001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.529408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.529696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.530396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.530711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.532531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982ce7d-a789-4db9-85e3-a96ad5e2c6d2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.545208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 409/802] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.546403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31a7ae-660f-4337-8464-e42cac14d611 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.546688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31a7ae-660f-4337-8464-e42cac14d611 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.546962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31a7ae-660f-4337-8464-e42cac14d611 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.548366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 410/803] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:35.548876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31a7ae-660f-4337-8464-e42cac14d611 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.620516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8478ede-a0e2-40e9-a150-b2c5bc397d2f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.659931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.660561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.660838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.661106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.661762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.662069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.663505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.681149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 394/804] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.681149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.775537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31a7ae-660f-4337-8464-e42cac14d611 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.777133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.793764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724cacc-ac45-431f-8a5e-0a3e3c03de8f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.812369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.813254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.813538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.813813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.814458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.814763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.816158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563fdcb-21e9-4767-a0aa-bbcf303b4a52 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.817902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.834526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 411/805] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.834745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.834745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.834903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.834982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.835064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.835146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/543542d17db0495f8151e779e6c6218f/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.835248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.836989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=543542d17db0495f8151e779e6c6218f, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.838424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.852130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:35.853318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:35.863418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 395/806] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:35 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/543542d17db0495f8151e779e6c6218f/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:35.863418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.953149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 412/807] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:35.954063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.036369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ae6fd-1896-4b98-81bf-bafbdbe857b0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.094308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.095476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.096021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.096513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.105410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.107927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.117846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc962f-ff36-41bd-b819-ff4d49e6e628 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:36.124069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.125017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:36.125968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:36.128548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 396/808] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.134668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.145679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.146246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.166786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.167852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.168337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.169327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.177258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.178977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:36.192182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.193157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:36.194214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:36.196740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 413/809] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.202580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.271929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec987c-4c64-469c-8e50-b72586b515ff None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.314721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.315737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.316220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.316735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.317665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.318235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.319958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.334025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590da53a-d573-4aac-9483-eb69cca7d8bb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.340175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 397/810] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:36.346794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.375703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.376655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.377108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.377559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.378416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.378995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.380802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.400799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 414/811] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.401983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.426862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.452634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.469427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b6b68-e902-4c86-8251-ca72b398e5f6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.512153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9fd8b-b92c-44f8-8fc9-4377e43fa5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.512793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9fd8b-b92c-44f8-8fc9-4377e43fa5ca None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.513065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9fd8b-b92c-44f8-8fc9-4377e43fa5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.513332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9fd8b-b92c-44f8-8fc9-4377e43fa5ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.521280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888494b2-460f-4166-89d6-2a7c65b807b9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.558935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.559932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.560374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.560647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.561230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.562593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.572870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.573860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:36.574691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 415/812] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.575562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.593937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9fd8b-b92c-44f8-8fc9-4377e43fa5ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.595413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9fd8b-b92c-44f8-8fc9-4377e43fa5ca None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.659629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.676734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68ef0c-f7ca-4142-804e-24767c4a6825 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.683638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.706287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 398/813] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:36.706287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec5c0b-d7d3-4c01-ba45-9f8387b85abf None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.706287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec5c0b-d7d3-4c01-ba45-9f8387b85abf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.706287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec5c0b-d7d3-4c01-ba45-9f8387b85abf None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.706287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 399/814] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:36.706287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec5c0b-d7d3-4c01-ba45-9f8387b85abf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.719779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.720419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.720695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.720963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.731664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.733112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:36.747600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.748567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:36.749485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:36.749485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:36.749485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:36.749485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 416/815] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.750192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.817460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec5c0b-d7d3-4c01-ba45-9f8387b85abf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.852172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a321-a65c-4ceb-8e28-db44c7c53768 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.852523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.853127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.853414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.853713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.854284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.854580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.855957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac1750-5f16-44f1-a61e-b4e22f7ab5c0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.864899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 400/816] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.864899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.864899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.864899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.880638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:36.881920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:36.894308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.895325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.895758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.896374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.909005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.910724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.911039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:36.926868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.928102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:36.929195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:36.929195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:36.929195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:36.935091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 417/817] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.935091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.944186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.979241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 401/818] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:36.979241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.048214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44010c-12bf-4729-9277-5d65af9db459 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.094261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.095037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.095653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.096002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.097195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.097802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.099878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.105698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951aa18-3853-452f-8d7f-88d86681883f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.119440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 418/819] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.121033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.151026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.151807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.152187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.152588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.153353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.153755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.155304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.161042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 402/820] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.162206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.194307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.211137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.244987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33306665-3435-4b30-a29f-42c3d02b9f08 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.285542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.286043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.286323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.286592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.287356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.287701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.288939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.292761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18d68e-5351-4a8a-99b1-eab1fa391208 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.304764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 419/821] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.304764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.335450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.336337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.336611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.336887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.345550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.347418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:37.359502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:37.360396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 403/822] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:37.361276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.411821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b5747-0267-48a7-91ad-ae3fdee12a6a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.441566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.448933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.449730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.450464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.450718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.451486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.452911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.487551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 420/823] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:37.489215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.567471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.595010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc64e7-5387-4c5d-addd-6cdcb75424e0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.626355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f23f40-a8cd-407e-8d8e-812a0a0d6dd6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.646955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.647694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.647985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.648255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] PATH_INFO: `/v3/services/network` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.648851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.651057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:37.664462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:37.665711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:37.666738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 404/824] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/services/network => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:37.667723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.672938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.672938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.672938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.672938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.678559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.680727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:37.693787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:37.695911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 421/825] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:37.698357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.751232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.789209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e126e-84c8-4810-a6a4-34f66eb1487e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.823483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c2613-1f28-4769-a7da-acc01361ae11 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.825752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.826436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.826772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.827114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.827887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.828276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.829779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.842909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 405/826] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.848532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.858925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.868348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.869466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.869949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.870661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.871919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.872487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.874101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.881765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 422/827] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.883140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.967443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7848f-7127-44cb-aa71-bfa4fc68bf2b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.990544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de83b30-fbab-48fd-b247-1b174966a0d3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.007193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.008125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.008838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.009314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.010204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:38.010692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.012387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.015629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.025888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.026631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.026994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.027373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/1e8df64cf5134a7e957de60145c8783b/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.029958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 406/828] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:38.031007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.044314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=1e8df64cf5134a7e957de60145c8783b, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.046009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.064413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 423/829] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:37 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/1e8df64cf5134a7e957de60145c8783b/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:38.072281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.109124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.152754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16674b4a-48d1-48ec-8cd9-6b04282a2b45 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.242315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.242944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.243233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.243513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.245314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] RBAC: Adding query filter params (type=network) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:38.245621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.247665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.262625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 407/830] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:38.272432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.303376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de380298-a59b-4e57-a5b9-3d9eded5fe41 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.354020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.355110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.355657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.356666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.357629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=ff1a6a40166e4f51a14cca9cd510e680) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:38.358271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.359883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a0852-a56d-4b8a-9870-68417f0e7f9b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.366710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 424/831] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/role_assignments?user.id=ff1a6a40166e4f51a14cca9cd510e680&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:38.368057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab901dd-991c-42fa-8acc-009c641cb88c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.368452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab901dd-991c-42fa-8acc-009c641cb88c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.368933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab901dd-991c-42fa-8acc-009c641cb88c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.370547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 425/832] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:38.384527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.384963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.385339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.412812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:38.414351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:38.421976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf82c7-3ef8-4677-8fbc-5a99f310d0ca None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.439714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.460709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.468614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.469248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.469559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.469804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.473784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.473784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.521081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 408/833] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:38.557594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 426/834] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:38.868985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.002220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312be3-ef52-41e3-a3b0-fb284ff9b7c8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.051789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.052438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.052715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.052983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.069192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:39.075622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:39.076544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:39.077393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:39.080051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 409/835] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:39.086149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.193621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99b5e4-c629-489f-a97c-c16aede41a8c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.233714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.234717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.235228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.235719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.236588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:39.237128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.238709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:39.248391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 427/836] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:39.254352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.306797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.367957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0abdb-8e9e-487c-98fa-552ba6c13c18 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.405014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.405650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.405928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.406199 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.418274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.420495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:39.435602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:39.436596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:39.437472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:39.437472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:39.437472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:39.437472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 410/837] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:39.443944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.456190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.573473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1c749-a6cd-46d8-b116-ac238bec0f5d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.613088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.614093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.614577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.615069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.615990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:39.616563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.618141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:39.631614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 428/838] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:39.637790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.756739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9286e0-c65b-43a4-85ed-7c08db8d5a01 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.759110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.796958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.798024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.798530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.799024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.807058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.808825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4c280ed8-eb9a-4e5e-80f0-61cd254d2881 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:39.820820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:39.821753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:39.822691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:39.825225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 411/839] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:39.831051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.876854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8cea1-13a6-47eb-a311-9d4c683b9761 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.877731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8cea1-13a6-47eb-a311-9d4c683b9761 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.878592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8cea1-13a6-47eb-a311-9d4c683b9761 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.880913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 412/840] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:39.886161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.886672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.887933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.910290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.912901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:39.915104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:39.947146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed407b-721f-4101-9f92-0ec86b5e1d05 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.984302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.984914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.985187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.985457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.986198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:39.986991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.988133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.008219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 413/841] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:40.015634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 429/842] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:40.018028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.113630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.127269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86c15a-422c-4c8f-927d-9d99acac906d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.162771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.163426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.164016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.164291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.233928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.235431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.299889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.330339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.346152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 414/843] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:45:40.352140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.443779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83197a0-e499-44cd-9693-e730b9c1d16c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.464980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416ab56-3229-4609-b7b5-08e947585f42 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.495026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.506508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.507530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.508161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.508799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.509127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.509438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.509720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.509999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.510786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.512517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.521784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e9209a25-2e4d-4bc1-924c-8df17af2fb2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:40.523446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:40.524441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:40.525280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:40.526018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.526090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 430/844] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:40.526686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b16e38-905c-4b00-8702-6d79223264f1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.526686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b16e38-905c-4b00-8702-6d79223264f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.526841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b16e38-905c-4b00-8702-6d79223264f1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.528339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 431/845] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:40.530364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b16e38-905c-4b00-8702-6d79223264f1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d1e857ed-1cf4-486a-b025-03b14f39c602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:40.545948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:40.546946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:40.547854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:40.547854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:40.547854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:40.547854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 415/846] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:40.548531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.548815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.549085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.572240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:40.573348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:40.608543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.646811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 416/847] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:40.648099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.659069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b16e38-905c-4b00-8702-6d79223264f1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.706987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.707950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.708966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.709243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.709873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.710175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.711588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.720872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 432/848] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:40.721879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.765222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48805184-f138-4d7b-9395-4bbc44837b61 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.798356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.804577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.805255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.805538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.805869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.807277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.807585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.808974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.825116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 417/849] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:40.833112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.862773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be288f20-278c-4ca5-95b8-10c6d43b19c8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.877492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.901881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.902588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.903875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.903875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.904575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.904947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.906442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.919925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 433/850] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:40.924621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.977528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed941-d4f4-4575-87c8-04f0437053db None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.017638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.018588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.018867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.019141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.027021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.029546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-009681eb-1216-44fd-bea8-c20c78be8682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:41.041894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.043095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:41.044667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 418/851] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:41.045304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c3231-89ef-4a67-9572-25157a234dfd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.045304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c3231-89ef-4a67-9572-25157a234dfd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.045440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c3231-89ef-4a67-9572-25157a234dfd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.046866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 419/852] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:41.051346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c3231-89ef-4a67-9572-25157a234dfd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.057360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b7a41-0747-47ed-8177-f5a1bdbd3a8b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.077039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.098193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43147535-5eba-4858-9f9e-6efad2c75e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.098830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43147535-5eba-4858-9f9e-6efad2c75e95 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.099104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43147535-5eba-4858-9f9e-6efad2c75e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.099405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43147535-5eba-4858-9f9e-6efad2c75e95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.111946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43147535-5eba-4858-9f9e-6efad2c75e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-43147535-5eba-4858-9f9e-6efad2c75e95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:41.117610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.118561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:41.119450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 434/853] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:41.127248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.127248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.127248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.142514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:41.143903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:41.162917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.177628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c3231-89ef-4a67-9572-25157a234dfd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.213245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 435/854] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:41.214429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.226648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.227437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.227838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.228214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.229010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.229515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.231897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.251342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 420/855] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:41.253085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.326043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fa217-8014-40b6-8cd6-cd038a6aae05 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.335700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.363751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.364798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.365363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.365814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.366665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.367255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.368978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.382910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 436/856] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:41.384038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894be99-59c7-4797-a6e8-aaa58d40bdfe None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.384621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.428147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.428948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.429332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.430118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.431017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=543542d17db0495f8151e779e6c6218f) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.431869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.434137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.447307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 421/857] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/role_assignments?user.id=543542d17db0495f8151e779e6c6218f&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:41.488197 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.506757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.514130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510d56b-5689-4638-9ad0-805e3f81465e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.555763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.557426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.558181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.558661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.575232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.576729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:41.595676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.596734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:41.597641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:41.597641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:41.597641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:41.602065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 437/858] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:41.603114 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.620959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9361-98c4-476b-9662-78e9662ffbd3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.648157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.665659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.666664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.667151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.667649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.668491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.670247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.682521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.683587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:41.685025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:41.685760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 422/859] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:41.692148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.761637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3938-6d7f-4fe5-8c56-84dd1d7b0aaa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.785793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.808846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646926-7238-43b2-85e1-00427663757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.809976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646926-7238-43b2-85e1-00427663757f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.810099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839eb6e-8613-48f8-9ef1-0dc77d5d96ec None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.810248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646926-7238-43b2-85e1-00427663757f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.810505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646926-7238-43b2-85e1-00427663757f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.811136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646926-7238-43b2-85e1-00427663757f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.811467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646926-7238-43b2-85e1-00427663757f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.814246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646926-7238-43b2-85e1-00427663757f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.831343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 438/860] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:41.837327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646926-7238-43b2-85e1-00427663757f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.855429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.856011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.856292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.856568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.857199 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.857503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.858875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.863944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 423/861] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:41.869285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.954755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.983266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646926-7238-43b2-85e1-00427663757f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.025339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663c88a-a092-46c7-8104-13546edac6a4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.037530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.039074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.039852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.040327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.051836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.054417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.066822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.067573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.067922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.068284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.069009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.069379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.070817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:42.073659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:42.074685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:42.075682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:42.081646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 439/862] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:42.084209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.084479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 424/863] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.212821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27562073-7761-489b-9c40-71ba55fede07 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.253193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.253819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.254146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.254488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.255264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.255579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.257072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.273963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.278645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 440/864] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.284389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.296462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.392326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7645fa-a44c-4ffa-a64c-63f7b95ce0f7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.428598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.429579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.430060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.430541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.506624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.508113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.516328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.615631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 425/865] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:42.621195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.734004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b956-12de-4c5d-baca-839d169a09e5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.770166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.770797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.771074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.771418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.782649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.784160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:42.799695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:42.800718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:42.801608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:42.801608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:42.801608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:42.801608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 441/866] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:42.806792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.913354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8f70a-884d-4b4b-aa46-c90e40aa5cab None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.931948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.949259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.950984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.951942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.952901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.953812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.954552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.956317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.970068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 426/867] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.975745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.030854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.086996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e41036-c33f-4754-8588-e89f7f25150d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.122703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.123388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.123709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.123985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.131025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.132524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-661f17a5-33d1-4339-b016-9ad63b1fb2b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:43.144401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:43.145325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:43.146211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 442/868] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:43.151577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.171673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380af137-f29a-416b-a86f-566d04086100 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.172015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380af137-f29a-416b-a86f-566d04086100 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.172189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380af137-f29a-416b-a86f-566d04086100 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.173613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 443/869] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:43.178783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.179067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.179380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.198891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:43.200243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:43.269140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eff30-fc8c-4a3e-aace-7393337a7bfe None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.283678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 444/870] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:43.304174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.304879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.305228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.305567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.306324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:43.306699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.308228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.322102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 427/871] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:43.328328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.378023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.437997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc630c-8f4f-421b-807a-a3d1b5f0a798 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.474504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.474504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.474504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.474903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.476682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=1e8df64cf5134a7e957de60145c8783b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:43.476682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.477407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e728a28-e68a-49b1-a55b-0889cc42fb6f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.484809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 445/872] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:43 2022] GET /identity/v3/role_assignments?user.id=1e8df64cf5134a7e957de60145c8783b&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:43.570580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.602235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.690945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea4144-2f1b-4a39-a198-062cfe7a40cc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.728147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557598e-ad20-4af1-becf-cc27b905bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.728756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557598e-ad20-4af1-becf-cc27b905bafc None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.729038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557598e-ad20-4af1-becf-cc27b905bafc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.729311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557598e-ad20-4af1-becf-cc27b905bafc None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.729967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557598e-ad20-4af1-becf-cc27b905bafc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.731352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557598e-ad20-4af1-becf-cc27b905bafc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.749161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 428/873] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:43 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:43.777437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a5eb-0604-434d-9bba-f19b748ddd90 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.778624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a5eb-0604-434d-9bba-f19b748ddd90 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.778624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a5eb-0604-434d-9bba-f19b748ddd90 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.780327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 446/874] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:43.786413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.786764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.787050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.807656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:43.809080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:43.897000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 429/875] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:44.169507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0a5eb-0604-434d-9bba-f19b748ddd90 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:44.281561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0a5eb-0604-434d-9bba-f19b748ddd90 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.291554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:44.320423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.321070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.321349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.321624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.322213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.323624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1a2265e5-a4e2-4822-9cfe-e060165dd474 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:44.333493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:44.334549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:44.335421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 447/876] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:44.341591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:44.359176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:44.452256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d7cb1-08d3-4405-a2b9-e93839e9c490 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.464731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a92c6-0eb6-4fb8-a7a1-ca2a5856d9ef None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.465060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a92c6-0eb6-4fb8-a7a1-ca2a5856d9ef None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.466564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a92c6-0eb6-4fb8-a7a1-ca2a5856d9ef None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.468536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 448/877] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:44.475854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.476138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.476411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.491249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.491929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.492210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.492482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.493171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:44.493490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.495772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.502690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 430/878] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:44.505518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:44.507351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:44.510083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:44.603735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 449/879] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:44.606091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:44.630681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45fb37-b090-4f96-a774-1b38aa133240 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.688670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.689844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.690357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.690851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.691762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:44.692296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.693995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5cf3e-0f57-4a13-95fe-388de4e633ad None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.705686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 431/880] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:44.925335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:45.034060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:45.037017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e247581-10c1-482d-a06f-5d6fb0c15e92 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:45.075355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08329136-52a2-4829-9a78-c4f1181fa866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:45.075973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08329136-52a2-4829-9a78-c4f1181fa866 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.076254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08329136-52a2-4829-9a78-c4f1181fa866 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.076527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08329136-52a2-4829-9a78-c4f1181fa866 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.077186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08329136-52a2-4829-9a78-c4f1181fa866 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:45.078563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08329136-52a2-4829-9a78-c4f1181fa866 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:45.099269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 450/881] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:44 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:46.011574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac000b-2f4a-4e51-af6b-7804d944df13 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.013341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac000b-2f4a-4e51-af6b-7804d944df13 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.013865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac000b-2f4a-4e51-af6b-7804d944df13 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.015720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 432/882] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:46.021629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.022525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.023029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.042405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:46.044632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:46.051271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cbe0-a73a-4a5b-90c4-390d28d2cfaa None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.051271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cbe0-a73a-4a5b-90c4-390d28d2cfaa None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.051271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cbe0-a73a-4a5b-90c4-390d28d2cfaa None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.052886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 433/883] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:46.059602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.060236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.060706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.080504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:46.082205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:46.147245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 451/884] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:46.164392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 434/885] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:46.420376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.438418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.513388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.515328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.535602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf964ded-a148-47ca-b8f3-b7e6c034db9d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.546375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5cfbf-2d23-4045-99a5-4db7979fae11 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.574123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.575270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.575746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.576230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.583387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.584394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.584822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.585243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.585998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.587530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.588937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:46.594354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:46.595321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:46.596166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:46.596948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:46.598043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:46.599149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:46.605565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 452/886] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:46.607279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 435/887] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:46.610078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.612294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.740220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0431-ec42-45a8-bdf3-72fdffe153d3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.752920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132068c-3eb2-4fad-9fa8-8cfe4898c440 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.779777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.780777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.781257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.781742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.782486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.782973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.784579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.786330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.794100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 436/888] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:46.800247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.803098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.803760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.804042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.804340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.804976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.805364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.806663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.816498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 453/889] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:46.824013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.842306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.931020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac223eee-c05b-4276-a45c-52894b3e4764 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.976505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.977275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.977625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7b0e6-d52e-4ae4-9d01-1ee32e2b3e3e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.977747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.978075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.978858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.979291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.981884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.995635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 437/890] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:46.998174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.020771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.021691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.022563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.022986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.036010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.037924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:47.054887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:47.055926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:47.056817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:47.056817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:47.056817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:47.064264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 454/891] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:47.065642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.075763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.124152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa083e-3c4a-4053-b2b3-5d8b9c228c06 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.129849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.169617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.170746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.171321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.171734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.172651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] RBAC: Adding query filter params (interface=public, service_id=fd5fa33ca9384fe2be92a9ab1d4dce4f, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.173100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.174595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdaac4-0cd9-41d6-920e-095f00eeff46 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.174775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7cec7-a935-4855-9047-02b31c3e92e4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.185992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 438/892] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/endpoints?service_id=fd5fa33ca9384fe2be92a9ab1d4dce4f&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:47.187251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529519e-f07e-4d55-957e-70515e09e879 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.187417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529519e-f07e-4d55-957e-70515e09e879 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.188666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529519e-f07e-4d55-957e-70515e09e879 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.189770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 439/893] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:47.193470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529519e-f07e-4d55-957e-70515e09e879 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.213758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.214374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.214653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.214925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.215608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.215911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.217313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62010b-aaa6-43e6-a040-a4f3190c7f27 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.231258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 455/894] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:47.231667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.231953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.232223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.251452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:47.252591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:47.319465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529519e-f07e-4d55-957e-70515e09e879 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.327025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 456/895] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:47.328152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.353679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.365653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e4fe5-41ad-4501-99e6-71593f7cfb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.367127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4fe5-41ad-4501-99e6-71593f7cfb18 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.367790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4fe5-41ad-4501-99e6-71593f7cfb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.368431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e4fe5-41ad-4501-99e6-71593f7cfb18 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.369252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4fe5-41ad-4501-99e6-71593f7cfb18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.371020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e4fe5-41ad-4501-99e6-71593f7cfb18 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.378537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 440/896] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:45:47.385266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47a4b2-81d7-4e08-8a54-58d302a7f98c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.385556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47a4b2-81d7-4e08-8a54-58d302a7f98c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.385837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47a4b2-81d7-4e08-8a54-58d302a7f98c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.387476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 441/897] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:47.392954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.393244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.393515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.399484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.416377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:47.417612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:47.482874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b19c06-0a55-4436-b60f-e14f6d95d019 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.515889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 442/898] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:47.524880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.525593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.525871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.526195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.534881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.536491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:47.550823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:47.551955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:47.552945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 457/899] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:47.560145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.601688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.672004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8877c-fdad-4d84-9c25-be6153eb6cd2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.696281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.716343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.716956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.717237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.717511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.718313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.718659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.720819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.746940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6756f1-8bfb-4a0a-9678-386d78ece058 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.747316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 443/900] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:47.753342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.762094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.790066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.791003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.791511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.791979 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.792824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.795256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.812405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 458/901] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:47 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:47.813533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.898268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fb2df-022f-4a02-9187-06afd2e3a49d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.956999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.957925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.958279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.958575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.962527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef22ce-23cb-4cc7-9335-6574d3a637d2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.020683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.022941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.023642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.023934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.024243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.024834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.026350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:48.040150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:48.041279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:48.042385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 459/902] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:48.049752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.060028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.061623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.096973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.176264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1435a-6076-4c9d-a1dd-af65ab211523 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.200618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 444/903] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:48.210383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.215538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.216330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.216729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.217084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.217926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.218321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.219897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.230507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 460/904] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:48.232044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.293732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.328562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20874d9a-98fa-4d68-ab01-0ffa5ad04e98 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.335434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.344589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85483c96-36fb-4de8-b664-92dd868c6076 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.372916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.373844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.374335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.374686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.386397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.387507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.387794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.388069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.388706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.389016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.390494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.390758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.393258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.401008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 461/905] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:48.404188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:48.412736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:48.413720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:48.414627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:48.414627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:48.414627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:48.414627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 445/906] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:48.420420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.523097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f81bb-639d-47ec-b8a7-cf905429a425 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.523900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2ccbd-87b9-424f-a135-ec9f5b9a5708 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.551316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.561797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.562306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.562465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.562695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.562950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.563725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.564016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.564133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.564600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.565113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.565429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.566227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] RBAC: Adding query filter params (interface=public, service_id=cf84f56461364da89632ca41d62d411a, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.566789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.568514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.579396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 462/907] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/endpoints?service_id=cf84f56461364da89632ca41d62d411a&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:48.583535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 446/908] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:48.584491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.584690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.612425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.706238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed74774-672e-43ee-a0ab-60b85641c911 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.711221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48b9f-c672-4973-b839-17a005f10d59 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.746369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.747408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.747911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.748383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.749234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.751053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.754721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.755561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.755952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.756235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.764668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.765378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 447/909] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:45:48.766161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:48.779948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:48.780905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:48.781899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 463/910] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:48.787482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.846983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.906869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e1996-d9f2-4504-919a-628e359bc9b3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.946166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.946794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.947132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.947444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.948149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.948455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.950568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeebf5-d468-48ef-ad88-2baf82b1332d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.963637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 448/911] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:48.970302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.064769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:49.088078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289f162-adb1-4045-b56b-0f4a7f644d61 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.125782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.126749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.127257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.127734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.128670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] RBAC: Adding query filter params (role.id=aebafcc9f64b42558dde9415d00cdedc, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=1e8df64cf5134a7e957de60145c8783b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:49.129260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.130921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42a3b-36b9-4af7-aed6-926f74b0f6fb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.138349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 464/912] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/role_assignments?user.id=1e8df64cf5134a7e957de60145c8783b&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=aebafcc9f64b42558dde9415d00cdedc => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:49.710984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92939199-ce31-4fd5-a026-1c474dcc2028 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.711837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92939199-ce31-4fd5-a026-1c474dcc2028 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.711837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92939199-ce31-4fd5-a026-1c474dcc2028 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.713214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 449/913] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:49.720197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd336a1-ff3f-40d8-a6d4-06bf08dc981f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.720513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd336a1-ff3f-40d8-a6d4-06bf08dc981f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.720794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd336a1-ff3f-40d8-a6d4-06bf08dc981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.740586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3cd336a1-ff3f-40d8-a6d4-06bf08dc981f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:49.741715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3cd336a1-ff3f-40d8-a6d4-06bf08dc981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:49.819311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 465/914] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:50.106565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92939199-ce31-4fd5-a026-1c474dcc2028 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.200309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334b58-3fd9-4119-b31a-b4103cdaa678 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.201264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334b58-3fd9-4119-b31a-b4103cdaa678 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.201751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1334b58-3fd9-4119-b31a-b4103cdaa678 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.203467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 466/915] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:50.208864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.209450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.209932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.215063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92939199-ce31-4fd5-a026-1c474dcc2028 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.226495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.229042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:50.231075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:50.258305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.259360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.259641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.259911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.261059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.262978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:50.275258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:50.276308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:50.277136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:50.281466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 450/916] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:50.282771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.323317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 467/917] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:50.388253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa1cd1-73d6-4837-be69-9dc72aa8560b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.423067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.423705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.423990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.424266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.424895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:50.425202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.426575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.431740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 451/918] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:50.437987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.446766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.548370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568afc1b-61b4-469c-ba0b-072ab0e7287c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.588904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.589518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.590355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.590634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.591306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:50.591613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.592986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.598269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 468/919] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:50.603788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.622344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.651003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.700054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.719900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abc437-35b5-4978-919f-cc98ddf73af5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.735498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894feca-f555-4f40-bef5-6612d6784c0a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.757697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.758721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.759101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.759591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.760294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.761994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.775324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.775932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.776401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.776690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.777785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.779243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.787027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 452/920] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:50.789134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:50.790149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:50.791021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 469/921] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:50.797076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.912267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.924499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028348e-5df0-44ab-8fef-cd814540e46e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.962911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.963929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.964480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.965489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.966329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:50.966852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.968484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.974103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 453/922] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:50.981223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.103376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4f776-5d7f-4dd7-bce9-96d7eadd0ef5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.140246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.140869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.141150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.141424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.142059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:51.142364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.143775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41700bae-f398-47e8-8e8c-1cf95ba5f329 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.145098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.151736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 470/923] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:51.158628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.208908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a11e3-2b91-4254-ba7e-d52d3d000089 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.210086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a11e3-2b91-4254-ba7e-d52d3d000089 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.210579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a11e3-2b91-4254-ba7e-d52d3d000089 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.213692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 471/924] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:51.218945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1f50-aa64-43d6-bdf2-8a4090fa476c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.219504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1f50-aa64-43d6-bdf2-8a4090fa476c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.219987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1f50-aa64-43d6-bdf2-8a4090fa476c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.239172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2a6c1f50-aa64-43d6-bdf2-8a4090fa476c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:51.240816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2a6c1f50-aa64-43d6-bdf2-8a4090fa476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:51.272508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c62f0-0774-4b2f-85f6-38f3311a2e3f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.311187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.311823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.312109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.312645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.313181 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] RBAC: Adding query filter params (interface=public, service_id=696a8da5f88045048e61f083d3db623f, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:51.313390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.314763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257477b-65a4-4001-8ee3-2203492f02d6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.323905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 454/925] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/endpoints?service_id=696a8da5f88045048e61f083d3db623f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:51.323905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7ebf-8bfb-4546-8a9f-01c0486cdfac None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.323905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7ebf-8bfb-4546-8a9f-01c0486cdfac None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.323905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7ebf-8bfb-4546-8a9f-01c0486cdfac None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.323905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 455/926] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:51.328118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7ebf-8bfb-4546-8a9f-01c0486cdfac None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.330234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 472/927] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:51.331789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.332149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.333283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.351515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.354292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:51.356548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:51.431844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 473/928] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:51.458732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7ebf-8bfb-4546-8a9f-01c0486cdfac None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.494042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.494645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.494919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.495185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.495805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.497783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.510034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 456/929] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:45:51.625332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.690448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.727597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.739922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45dbd8-3490-464d-829a-1569fa2626d7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.786562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.787598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.788094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.788592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.789400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.791037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:51.801172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:51.802213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:51.803127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:51.806115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 474/930] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:51.812440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.845688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.871898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7526a3f-4544-434e-890b-58fa13d57f8f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.910802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.911533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.911816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.912087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.923543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:51.929526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:51.930491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:51.931381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 457/931] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:51.936982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.939233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2fbf3-73ba-4c19-8f11-d33952cb7db6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.971086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.984310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2c445-2543-453f-be0a-76873266240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.985140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2c445-2543-453f-be0a-76873266240a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.985648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2c445-2543-453f-be0a-76873266240a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.986103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2c445-2543-453f-be0a-76873266240a None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.986872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2c445-2543-453f-be0a-76873266240a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:51.987356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2c445-2543-453f-be0a-76873266240a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.988872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2c445-2543-453f-be0a-76873266240a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.994370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 475/932] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:51.999756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2c445-2543-453f-be0a-76873266240a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.057822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9c7ff-0767-44ab-af37-b0f0a4e270ef None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.096147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.096757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.097031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.097300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.097958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.098259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.100612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.115130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 458/933] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:52.116543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.122071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.147877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2c445-2543-453f-be0a-76873266240a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.194531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.195816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.196325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.197019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.198772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.199879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.201472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.212951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 476/934] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:52.214561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.237602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cf40c-bc9c-46d1-9fd1-d0854b0d7913 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.279576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.280227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.280535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.280764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.283642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.295094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.296902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:52.313283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:52.314352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:52.315306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:52.315306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:52.315306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:52.317560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 459/935] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:52.322757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.346181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f46be-c557-449d-a7af-11a208650d36 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.381627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.382237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.382654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.382936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.383904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.385279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.397575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.412422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 477/936] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:45:52.456106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bc788-e0f4-4e79-9d3a-ba1a2c03ec11 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.503838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.504378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.504662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.504939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.505673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.505987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.507599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.521943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 460/937] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:52.528414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.645475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e88390-c867-4cc5-82cb-8b453fc26f4a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.653526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.685727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.686365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.686640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.686910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.695962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.697411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:52.709211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:52.710189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:52.711127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 478/938] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:52.716049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.818544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff39d9c-37d1-496e-ad76-bb2d33ea772e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.832899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.854945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.856221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.856495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.856770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.857495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.857796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.859237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.877953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 461/939] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:52.884622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.991662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c311cf8-a5a4-41ea-9264-af849b45b820 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.029673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.030337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.030616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.030894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.047023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.048538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-29c606b1-a10f-456b-8dad-bf711d6dddc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:53.064354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:53.066507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:53.068699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:53.068699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:53.068699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.068699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 479/940] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:53.071582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.140967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d778-05ce-4d5b-ab55-a9015e7063ef None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.141292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d778-05ce-4d5b-ab55-a9015e7063ef None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.141494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028d778-05ce-4d5b-ab55-a9015e7063ef None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.144189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 480/941] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:53.148231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7477978-2966-47b1-a41f-470345226865 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.148510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7477978-2966-47b1-a41f-470345226865 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.148783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7477978-2966-47b1-a41f-470345226865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.167154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a7477978-2966-47b1-a41f-470345226865 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:53.168399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a7477978-2966-47b1-a41f-470345226865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:53.182795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.195342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd76fd-51c2-4128-87aa-6b0bd3c59247 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.240955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.241983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.242468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.242947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.243871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:53.244282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.245762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.260827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 481/942] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:53.267186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 462/943] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:53.276339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7477978-2966-47b1-a41f-470345226865 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.381968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7477978-2966-47b1-a41f-470345226865 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.417623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.418794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.419069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.419365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.426245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.427732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:53.439578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:53.440466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.441365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 482/944] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:53.446818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.519598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.555942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af029d5f-e12f-49ba-b54f-f5aa3b9af165 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.568153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.580163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.602420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.603039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.603342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.603611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.604300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:53.604602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.607324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.621796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 463/945] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:53.630300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.696294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfa3a5-b88c-4c8f-87d2-3f553b76c9f2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.732985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.744260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981e09e-d04a-465a-8854-4ff6933319b7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.745527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.746587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.747954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.748382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.749181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.750782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.754132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING keystone.server.flask.common [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 14:45:53.780227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.sql.core [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 14:45:53.780227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 14:45:53.780227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [parameters: {'id': 'a9e6b1e332df4d9fa7d5042615260f0d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 14:45:53.780227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 14:45:53.783369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb42dc-6e48-4e97-9c5a-a6093d058fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.784195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb42dc-6e48-4e97-9c5a-a6093d058fe2 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.784564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb42dc-6e48-4e97-9c5a-a6093d058fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.784962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb42dc-6e48-4e97-9c5a-a6093d058fe2 None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/1e8df64cf5134a7e957de60145c8783b/roles/aebafcc9f64b42558dde9415d00cdedc` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.787289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb42dc-6e48-4e97-9c5a-a6093d058fe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=1e8df64cf5134a7e957de60145c8783b, role_id=aebafcc9f64b42558dde9415d00cdedc)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.788951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb42dc-6e48-4e97-9c5a-a6093d058fe2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-f4580790-bbf7-4b84-b827-690da74071fd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a9e6b1e332df4d9fa7d5042615260f0d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a9e6b1e332df4d9fa7d5042615260f0d. Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:53.789409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 22 14:45:53.791193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application yield resource Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise self.value Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application session.commit() Sep 22 14:45:53.794023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.flush() Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application compat.raise_( Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 14:45:53.795954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 14:45:53.797456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:53.798584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [parameters: {'id': 'a9e6b1e332df4d9fa7d5042615260f0d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.799733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.801749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a9e6b1e332df4d9fa7d5042615260f0d. Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 483/946] 10.222.0.66 () {60 vars in 1140 bytes} [Thu Sep 22 14:45:53 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96ce8-5e7b-4c91-b3dc-c90b42ff1ec6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.803858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96ce8-5e7b-4c91-b3dc-c90b42ff1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.805174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b96ce8-5e7b-4c91-b3dc-c90b42ff1ec6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.805174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 484/947] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:53.805174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96ce8-5e7b-4c91-b3dc-c90b42ff1ec6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.812748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 464/948] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:53 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/1e8df64cf5134a7e957de60145c8783b/roles/aebafcc9f64b42558dde9415d00cdedc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:45:53.813897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.814267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.814617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.838488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:53.840431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:53.892363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.943269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 465/949] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:53.955046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b96ce8-5e7b-4c91-b3dc-c90b42ff1ec6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.996845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.997849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.998266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.998673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.006836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.008990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:54.021750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:54.023315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:54.024696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 485/950] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:54.030868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.118152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.141018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f929dff-82c3-4d4f-b25b-931428e56c19 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.177954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.178570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.178845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.179113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.179853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:54.180092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.181512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.199545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 466/951] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:54.226666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.353878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94252d5f-d1b7-43bc-a657-f2e29d6b6f4a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.382266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.395534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.396532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.397015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.397514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.398106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.399515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:54.408782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:54.410051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:54.410976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 486/952] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:54.416082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.533710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95454d8a-f80c-4cc0-adcc-9758136d3477 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.569628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.570564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.571015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.571486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.572300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:54.572786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.575117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.581313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 467/953] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:54.587108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.702611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a18e4d-8b81-4370-afd9-00f130b24aa9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.738968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.739952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.740404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.740851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.741679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:54.742167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.743914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28b55-4381-4140-af77-1b36f725d721 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.751168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 487/954] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:54.763243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.763243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.872450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdb3f0-0282-441e-8347-af8fa233324a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.873390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3e04a-eb5e-445c-ba5d-2bb8c46d7522 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.873680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3e04a-eb5e-445c-ba5d-2bb8c46d7522 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.873954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3e04a-eb5e-445c-ba5d-2bb8c46d7522 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.876626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 488/955] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:54.881017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.881296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.881572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.900927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:54.902558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:54.909868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.910533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.910819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.911089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.911890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.913260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.937501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 468/956] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:54.961948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:55.000162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 489/957] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:55.279113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.353896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:55.397429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1d36d-35cc-46bb-a28f-f1e07a04bdfc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.432558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.433190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.433497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.433767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.434350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.435739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:55.444603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:55.445618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:55.446729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 469/958] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:55.454014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.571016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e700c8-b67f-456c-9d83-da238519d550 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.608023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.608640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.608922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.609197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.609831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:55.610140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.611556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb013f-27ef-43af-9b22-18db943986f5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.617030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 490/959] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:55.625878 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.736742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393e4a-4a9c-4f97-a0f9-02f0fb7beb33 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.762724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:55.775182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.776075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.776520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.776946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.778281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] RBAC: Adding query filter params (type=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:55.778737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.780323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dd8f9-6742-41b2-88cb-12c7ee8ff4ae None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.785693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 470/960] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:56.127130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4977cc-c688-44df-b0a9-43a2edd99919 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.127533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4977cc-c688-44df-b0a9-43a2edd99919 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.127733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4977cc-c688-44df-b0a9-43a2edd99919 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.129319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 491/961] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:56.135084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.135588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.135667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.154585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:56.155810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:56.226829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 471/962] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:56.492684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4977cc-c688-44df-b0a9-43a2edd99919 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:56.540012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:56.612468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4977cc-c688-44df-b0a9-43a2edd99919 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:56.650252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1195e3-8375-4ddc-9665-bdf9f67395c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:56.650892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1195e3-8375-4ddc-9665-bdf9f67395c3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.651170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1195e3-8375-4ddc-9665-bdf9f67395c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.651475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1195e3-8375-4ddc-9665-bdf9f67395c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.667261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1195e3-8375-4ddc-9665-bdf9f67395c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5e1195e3-8375-4ddc-9665-bdf9f67395c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:56.672624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:56.673557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:56.674413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:56.676763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 492/963] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Sep 22 14:45:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:56.682551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:56.687555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4db802-e38a-4b52-90b0-23db5c4c3b1b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.687836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4db802-e38a-4b52-90b0-23db5c4c3b1b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.688111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4db802-e38a-4b52-90b0-23db5c4c3b1b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.690140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 493/964] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:56.698276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.698560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.698642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.718697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:56.719942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:56.799224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 494/965] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:56.815794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7d921-fa44-4f08-8aec-853c6c851fcf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:56.835058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:56.854539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:56.855183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.855494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.855764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.856424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:56.856724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:56.858218 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:56.867054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 472/966] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:45:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:56.872997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:56.909833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:56.985070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3866e55-11ff-45b5-8775-90d65b461499 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.021460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.022440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.022926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.023428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.034056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.036008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:57.056644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:57.057673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:57.059415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:57.059415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:57.059415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.062829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 495/967] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:57.071984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.103870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.217023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6819800-240d-44c2-bcf6-3f2b7681ff4c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.236904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.252366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.252979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.253261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.253360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9bd9-e2d3-4866-8be0-04cb5b106899 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.253533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.254174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:57.254480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.256297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa7e6d-c122-4455-9f47-5704a1a059eb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.274249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 473/968] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:57.274249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d712330-4320-4295-bec8-73d837e22ad2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.274249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d712330-4320-4295-bec8-73d837e22ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.274249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d712330-4320-4295-bec8-73d837e22ad2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.274249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 474/969] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:57.275823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d712330-4320-4295-bec8-73d837e22ad2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.299828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.300676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.300957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.301226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.301875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.303327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.318481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.sql.core [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 14:45:57.318481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 14:45:57.318481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [parameters: {'user_id': '213f926b9f3649b2a1d8dbb84814f8d0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 14:45:57.318481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 14:45:57.324522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ba1b337c-06c4-4b71-8d19-3e328788f280 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:57.325589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:57.326580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application yield resource Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise self.value Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application session.commit() Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 14:45:57.332358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.flush() Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application compat.raise_( Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 14:45:57.333308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:57.334216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:57.335081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [parameters: {'user_id': '213f926b9f3649b2a1d8dbb84814f8d0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:57.337832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 22 14:45:57.338823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 496/970] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Sep 22 14:45:57 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.340606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.348414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:57.349915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:57.403264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d712330-4320-4295-bec8-73d837e22ad2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.426178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 497/971] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:57.427023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.444319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.444955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.445233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.445504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.453362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.454629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:57.467139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:57.468287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:57.469392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:57.473686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 475/972] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:57.474004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.546752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.578938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2ef0f-4371-4a98-833a-f6605a2b251d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.590578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.595080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be381331-feb5-4081-b15d-176f6d8a105a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.627391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.628030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.628312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.628585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.634944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.635839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.636532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.637085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.637505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.638195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:57.638684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.639397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.640693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:57.654531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:57.655589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:57.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:57.659364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 498/973] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:57.661230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.665644 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 476/974] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:57.672374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.785068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af589daa-d539-43f9-8084-4c265bd9128d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.785475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043819ea-0330-45e9-8f3d-2c612cbcb406 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.820790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.821401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.821685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.821946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.822645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:57.822952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.824958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.830791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc7ff-dafd-4f03-9e09-6e57fdd4771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.831440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc7ff-dafd-4f03-9e09-6e57fdd4771f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.831790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc7ff-dafd-4f03-9e09-6e57fdd4771f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.832133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc7ff-dafd-4f03-9e09-6e57fdd4771f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.835818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.864507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 499/975] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:57.865632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.876287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.930549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc7ff-dafd-4f03-9e09-6e57fdd4771f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.933967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc7ff-dafd-4f03-9e09-6e57fdd4771f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.037047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cac1a-f9d2-400a-aafd-4b0734c50edb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.083061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.083763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.084104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.084438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.085026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.086437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.087194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 477/976] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:58.088555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a222-cb7f-48cc-b2ec-62640054a5cf None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.088913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a222-cb7f-48cc-b2ec-62640054a5cf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.089257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a222-cb7f-48cc-b2ec-62640054a5cf None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.091167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 478/977] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:58.096150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a222-cb7f-48cc-b2ec-62640054a5cf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-fe38805b-3762-4fb4-ab28-66d9808d9a78 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.097517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:58.098686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:58.099764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 500/978] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:58.100652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.100652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.100652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.122359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:58.123596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:58.180253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.184214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.198231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 501/979] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:58.200175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.227756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a222-cb7f-48cc-b2ec-62640054a5cf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.264210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.264845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.265111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.265380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.277901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.279420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:58.294184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:58.295411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:58.296452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:58.296452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:58.296452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:45:58.296452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 479/980] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:58.300776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.315743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022d0c-8fe9-4c3f-9798-222580631d46 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.359368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.359990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.360337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.360697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.361824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.362197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.364121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.371441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 502/981] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:58.376872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.420315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312356c-cf65-464a-a042-97d2eba93a33 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.431856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.433184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.458314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.458924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.459219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.459527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.460171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.460478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.461847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.475302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 480/982] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:58.475532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.491390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1226c-60eb-4a76-9601-711ba0c9ff5d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.525473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.526071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.526342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.526606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.527869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.528169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.529524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.534128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 503/983] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:58.534893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.580217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b35ea-af4f-4a78-8b63-82cff48a64ee None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.615247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.615644 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.615921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.616193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.616876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.618224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.631954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298e20c-6baf-48ad-88be-b4db1398f967 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.632467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 481/984] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:58 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:58.633257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.651843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.668421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.687705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.688416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.688696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.688966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.697473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.699001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:58.712413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:58.713504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:45:58.714399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 504/985] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:58.719763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.753453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cec3d-6b03-451c-b237-596ad51ae15d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.792279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.793636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.793748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.793956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.794670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] RBAC: Adding query filter params (interface=admin, service_id=696a8da5f88045048e61f083d3db623f, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.794980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.796410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.806486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 482/986] 10.222.0.66 () {56 vars in 1249 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/endpoints?service_id=696a8da5f88045048e61f083d3db623f&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:58.808824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.843928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238e12d-b9a4-4b3b-abdf-b034cbcee618 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.881257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.882203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.882661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.883108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.884278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.884758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.886658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.900581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 505/987] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:58.906677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.922536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.930729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e76be-dcce-4dab-87db-ebe0dc7e1315 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.940068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.970206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a5181-7aab-43da-9ce4-19aa086aa79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.971408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a5181-7aab-43da-9ce4-19aa086aa79e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.971691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a5181-7aab-43da-9ce4-19aa086aa79e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.972137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a5181-7aab-43da-9ce4-19aa086aa79e None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.972562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a5181-7aab-43da-9ce4-19aa086aa79e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.973935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a5181-7aab-43da-9ce4-19aa086aa79e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.991227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 483/988] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:45:59.029073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d290d06-30b9-4147-aeef-f169b1771a04 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.068123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.068749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.069027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.069296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.070043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] RBAC: Adding query filter params (role.id=aebafcc9f64b42558dde9415d00cdedc, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=1e8df64cf5134a7e957de60145c8783b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.070348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.071932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3eadf-2462-4bb6-8210-5a722e686444 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.079964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 506/989] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/role_assignments?user.id=1e8df64cf5134a7e957de60145c8783b&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=aebafcc9f64b42558dde9415d00cdedc => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:00.832431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc09582-90b8-42bf-878b-3c09e58c41d0 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:00.832954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc09582-90b8-42bf-878b-3c09e58c41d0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:00.832954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc09582-90b8-42bf-878b-3c09e58c41d0 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:00.834459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 484/990] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:00.840184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186bf69-6559-4afb-9c6b-fbec29fe62bc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:00.840627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186bf69-6559-4afb-9c6b-fbec29fe62bc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:00.840733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186bf69-6559-4afb-9c6b-fbec29fe62bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:00.859918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3186bf69-6559-4afb-9c6b-fbec29fe62bc None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:00.861116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3186bf69-6559-4afb-9c6b-fbec29fe62bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:00.932448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 507/991] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:01.179082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3052b6d-f50b-447b-8fdb-26dc2a6f2af8 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.179472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3052b6d-f50b-447b-8fdb-26dc2a6f2af8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.179566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3052b6d-f50b-447b-8fdb-26dc2a6f2af8 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.181138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 485/992] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:01.187650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5cb94-6ba8-47aa-8af5-5535aa5d012f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.187960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5cb94-6ba8-47aa-8af5-5535aa5d012f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.188204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5cb94-6ba8-47aa-8af5-5535aa5d012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.207921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fda5cb94-6ba8-47aa-8af5-5535aa5d012f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:01.209271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fda5cb94-6ba8-47aa-8af5-5535aa5d012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:01.211643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3052b6d-f50b-447b-8fdb-26dc2a6f2af8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.230717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:01.280977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 508/993] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:01.297111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae100654-90b7-4e7d-880e-7f1ff470088d None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.297394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae100654-90b7-4e7d-880e-7f1ff470088d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.297676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae100654-90b7-4e7d-880e-7f1ff470088d None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.302873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 509/994] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:01.304368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81e689-d729-4898-9910-d25737d339bb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.304659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81e689-d729-4898-9910-d25737d339bb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.304940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81e689-d729-4898-9910-d25737d339bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.324263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bc81e689-d729-4898-9910-d25737d339bb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:01.325469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bc81e689-d729-4898-9910-d25737d339bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:01.328280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3052b6d-f50b-447b-8fdb-26dc2a6f2af8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.366371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.367015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.367316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.367591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.368341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.369549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:01.378654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:01.379747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:01.380585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 486/995] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:01.385891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.400022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 510/996] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:01.412164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:01.497704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e117c-faa1-431c-85f6-6968784e4358 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.532574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.533177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.533488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.533724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.534343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:01.534645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.536030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:01.540995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 487/997] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:01.546996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81e689-d729-4898-9910-d25737d339bb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.551581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.638477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:01.650917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:01.659319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81e689-d729-4898-9910-d25737d339bb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.673012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079d453-7fbb-4d44-92a0-d3dc04e23753 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.696368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.696986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.697256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.697524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.698142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:01.698442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.699829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:01.704563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 511/998] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:01.705476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.707809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.708442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.708722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.708996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.709819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.711186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:01.720721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:01.721944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:01.722786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 488/999] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:01.723649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.806083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6763fe-bdd3-42ab-968e-f7fce5f02108 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.823693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a5995-2d32-4782-b1a2-72d7789f34fa None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.840733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.841364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.841645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.841920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] PATH_INFO: `/v3/services/image` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.842510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.843914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:01.852668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:01.853723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:01.855247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 512/1000] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:01.856227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.861740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d94fd-e863-44be-9753-13434e136548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.862345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d94fd-e863-44be-9753-13434e136548 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.862618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d94fd-e863-44be-9753-13434e136548 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.862882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d94fd-e863-44be-9753-13434e136548 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.863607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d94fd-e863-44be-9753-13434e136548 None admin] RBAC: Adding query filter params (interface=public, service_id=82ad143ed6af4ab6bb7a43f6f06bc652, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:01.863911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d94fd-e863-44be-9753-13434e136548 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.865264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d94fd-e863-44be-9753-13434e136548 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:01.870427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 489/1001] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/endpoints?service_id=82ad143ed6af4ab6bb7a43f6f06bc652&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:01.871748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d94fd-e863-44be-9753-13434e136548 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:01.873152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:01.894572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:01.960940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb3e0e-9b8c-40e3-b06e-ee2ccf982615 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.977989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d94fd-e863-44be-9753-13434e136548 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:01.993485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:01.994091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:01.994374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:01.994643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:01.995291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:01.996081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:01.997453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.002131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 513/1002] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:02.002904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.012331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.012933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.013207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.013481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.014103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] RBAC: Adding query filter params (name=image) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:02.014404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.015783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.020571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 490/1003] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:02.021364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.088169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:02.098752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:02.111189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d5247-9f81-4084-b911-2a5fe76fe490 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.120539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79a291-f413-4758-abe5-867e1bafc95d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.145378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.145989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.146395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.146672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.147310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.148681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.154280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.154885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.155159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.155382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 514/1004] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:46:02.155490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.156098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:02.156185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.156401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.157756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.168400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 491/1005] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:02.176976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.289426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9760e89-8b54-47c5-a1ed-bcb9de7a3f68 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.305681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413f9ab-9e1f-4559-9c08-db9917eb03eb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.335557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.336178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.336458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.336735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.337368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] RBAC: Adding query filter params (type=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:02.337673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.339122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c7c36-0d4b-4577-b4ae-eb32c8a07108 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.346071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 515/1006] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:02.361099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a6e39-37b5-4d4b-8b24-19397ba4f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.362101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a6e39-37b5-4d4b-8b24-19397ba4f986 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.362101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a6e39-37b5-4d4b-8b24-19397ba4f986 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.362342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a6e39-37b5-4d4b-8b24-19397ba4f986 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.364958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a6e39-37b5-4d4b-8b24-19397ba4f986 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.366661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a6e39-37b5-4d4b-8b24-19397ba4f986 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.388827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 492/1007] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:46:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:04.458317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f66ed1-3c74-4757-b916-42baf0a9e342 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.458864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f66ed1-3c74-4757-b916-42baf0a9e342 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.458864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f66ed1-3c74-4757-b916-42baf0a9e342 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.460318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 516/1008] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:04.467032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d862-bcd2-4228-9f2d-97492386b71d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.467577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d862-bcd2-4228-9f2d-97492386b71d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.467677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40d862-bcd2-4228-9f2d-97492386b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.474466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:04.490232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6e40d862-bcd2-4228-9f2d-97492386b71d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:04.491594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6e40d862-bcd2-4228-9f2d-97492386b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:04.531016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b967d1-f810-4dbc-b2e1-f2d4b6a113f8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.531399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b967d1-f810-4dbc-b2e1-f2d4b6a113f8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.531627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b967d1-f810-4dbc-b2e1-f2d4b6a113f8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.533150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 517/1009] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:04.538010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.538279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.538553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.551340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:04.559137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:04.560431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:04.567678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 493/1010] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:04.590007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9cef-4636-4e51-b186-346ab3a472ef None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.590293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9cef-4636-4e51-b186-346ab3a472ef None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.590574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9cef-4636-4e51-b186-346ab3a472ef None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.591987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 494/1011] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:04.596986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.597275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.597548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.615766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:04.616971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:04.637912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 518/1012] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:04.683989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 495/1013] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:04.835898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:04.902506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:04.937821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef53c73-b24c-4148-870b-899f50e0e29c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:04.962636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:04.971642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:04.972275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:04.972548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:04.972810 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:04.973380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:04.974738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:04.977229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:04.984473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:04.985717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:04.986787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 519/1014] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:04.991549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.009983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bc54f-246d-4c05-b783-50dfd057bb43 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.045019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.045627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.045909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.046187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.046845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.048284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.063568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 496/1015] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:04 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:05.064376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.101773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731183ca-014b-4b84-8770-f174a3943fa1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.149273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.149903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.150226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.150505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.151786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.155143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.156430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 520/1016] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:05.162197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.184827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823197-9607-4d3e-b0c2-35670a0127ac None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.206222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:05.229208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.230383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.230674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.230951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.231629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:05.231937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.233342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.234262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:05.244289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 497/1017] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:46:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:05.245380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.285205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e69f-e438-428d-a796-a731c53041e4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.330326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.330948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.331250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.331561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.332630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.334802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.358974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa262c4-8850-48e3-a784-2bacb4fe64ea None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.367828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 521/1018] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:05 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:05.420344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.420983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.421262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.421541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.422218 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:05.422509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.424234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94449-3eb4-4921-8d08-0da52c9af0c5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.430694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 498/1019] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Sep 22 14:46:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:05.436914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.565993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36a8a5-3691-43da-a7bf-2ce7fd716f4c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.584947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:05.606235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a314a-d87d-498b-9e6c-c689bab53b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.606850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a314a-d87d-498b-9e6c-c689bab53b48 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.607127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a314a-d87d-498b-9e6c-c689bab53b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.607457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a314a-d87d-498b-9e6c-c689bab53b48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.608122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a314a-d87d-498b-9e6c-c689bab53b48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.609531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a314a-d87d-498b-9e6c-c689bab53b48 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.631285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 522/1020] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:46:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:07.280701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44440fdc-7cb0-41db-b84e-208589d6a99a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.281228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44440fdc-7cb0-41db-b84e-208589d6a99a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.281306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44440fdc-7cb0-41db-b84e-208589d6a99a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.282753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 499/1021] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:07.287771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36f1ad-b6d1-482e-839d-f2da02fd8a35 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.288178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36f1ad-b6d1-482e-839d-f2da02fd8a35 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.288343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36f1ad-b6d1-482e-839d-f2da02fd8a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.306615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2d36f1ad-b6d1-482e-839d-f2da02fd8a35 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:07.307814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2d36f1ad-b6d1-482e-839d-f2da02fd8a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:07.381999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 523/1022] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:07.627059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0983e-08db-417f-afcd-685590b020fd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.627419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0983e-08db-417f-afcd-685590b020fd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.627649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0983e-08db-417f-afcd-685590b020fd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.629145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 500/1023] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:07.634450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.635606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.635899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.649624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0983e-08db-417f-afcd-685590b020fd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:07.655314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:07.656517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:07.702479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:07.733809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 524/1024] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:07.762524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0983e-08db-417f-afcd-685590b020fd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.800586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.801166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.801438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.801700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] PATH_INFO: `/v3/services/image` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.802276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.803651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d2aa2dca-f457-49d5-b86f-426d04ecb845 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:07.812389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:07.813508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:07.814412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:07.818440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 501/1025] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:07.819710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:07.849503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:07.882740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fa0f8-d088-491b-8c8f-445622aaa123 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.882914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fa0f8-d088-491b-8c8f-445622aaa123 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.883215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fa0f8-d088-491b-8c8f-445622aaa123 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.884948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 502/1026] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:07.890572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.890858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.891132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.909839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:07.911041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:07.931982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5c49c-e5c3-4903-940d-50ac5aa1d79e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.970452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.971067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.971425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.971651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.972302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] RBAC: Adding query filter params (name=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:07.972597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.973972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.979803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 525/1027] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:07.982639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 503/1028] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:07.985316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.014476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.057402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.085169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.094522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f73ba3-56b7-40f1-ba22-131910f4657f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.122432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b47d9-4c32-4d0d-aefa-f3b15b939c44 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.131873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.132513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.132847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.133099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.134124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] RBAC: Adding query filter params (type=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.134351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.135846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.148125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 526/1029] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:08.149122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.171894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.172532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.172815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.173091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.174215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.175707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.177278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 504/1030] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:08.183577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.258725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4a26-f3d7-40e0-ac9d-ec4a196be25e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.288551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf304e-aff1-4635-87f4-5d4e94acc362 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.294373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.294979 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.295283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.295556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.296243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] RBAC: Adding query filter params (interface=public, service_id=40fee12532124e32b7c8a33896c7cef5, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.296612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.297987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.298405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.303055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 527/1031] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/endpoints?service_id=40fee12532124e32b7c8a33896c7cef5&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:08.304031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.325305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.329064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.329678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.329960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.330251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.330902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.332405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.354319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 505/1032] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:46:08 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:08.355707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.418397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0828ed-36db-4f2e-8332-ed17e5887fad None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.457230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.457888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.458170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.458978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.475490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:08.482885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:08.484096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:08.485113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 528/1033] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:08.488821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.500797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b001d00-32cb-4bba-9be9-851cbc078e79 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.540971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.541591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.541862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.542137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.543288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.545310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.553843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 506/1034] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/services => generated 1957 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:46:08.586950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.611749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385e8d8-b25f-4a62-a89c-64bc8dfb9656 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.651870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.652499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.652778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.653048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.653721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.654026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.655441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.667242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 529/1035] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:08.671295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.704513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.788450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433732-0e42-4e6e-a795-2c151f8987b4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.823011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.823668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.823943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.824206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.834891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.836436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:08.850887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:08.852079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:08.852984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:08.852984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:08.852984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:08.852984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 507/1036] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:08.858290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.972246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b7a3d-e32f-4adf-b33c-68ec52b4247b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.007432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.007990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.008511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.008793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.009479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:09.009752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.011690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.023837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 530/1037] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:09.029853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.111875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.132930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e97614-8164-4e7b-a2b8-a0026041f5ec None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.168493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.169541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.170037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.170384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.179806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.181267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:46:09.193336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:09.194354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:09.195348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:09.199569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 508/1038] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:09.200859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.203775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.315951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e54f92-4d98-47bb-b7e4-8c0377b42758 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.353978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.354897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.355038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.355233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.355960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:09.356262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.357687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.380302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 531/1039] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:09.381937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.499046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab444296-1495-426a-a2b3-0583be024d09 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.535162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.535799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.536083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.536357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.547558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.606546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.607931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.740096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 509/1040] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:46:09.742464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.836399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.883828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f435a8-453a-43db-89ec-6e1633991029 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.922471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.923355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.923841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.924264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.936195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.937654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:09.954287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:09.956720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:09.958401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:09.958401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:09.958401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:09.958401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 532/1041] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:09.966652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:10.016369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:10.077514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2fe3-c4b2-475f-9588-ed998fb6f40c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:10.116247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:10.116875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.117162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.117429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.118082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:10.118389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:10.119822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:10.143641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 510/1042] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:10.146909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:10.254026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459a693-b155-4c69-ac14-591125fc4685 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:10.267567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:10.290838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:10.291505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.291787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.292059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.299502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:10.300991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:46:10.312442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:10.313373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:10.314275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:10.315376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 533/1043] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:10.319524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:10.423858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:10.428110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f40c4e-6169-4166-be19-07d2da1de420 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:10.465527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:10.465767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.465977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.466205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.466902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:10.467233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:10.468687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a9745-f3a0-4a06-aba0-451544f5dbbf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:10.487186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 511/1044] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:10.493947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:10.602568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6e52-f96b-4f6a-8de5-7247f4b12263 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:10.637537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:10.638147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.638426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.638699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.639452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] RBAC: Adding query filter params (role.id=aebafcc9f64b42558dde9415d00cdedc, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=ff1a6a40166e4f51a14cca9cd510e680) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:10.639765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:10.641199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c30b1-b272-4926-9281-1907b927e3a6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:10.651260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 534/1045] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:10 2022] GET /identity/v3/role_assignments?user.id=ff1a6a40166e4f51a14cca9cd510e680&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=aebafcc9f64b42558dde9415d00cdedc => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:10.653800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fbe61-afea-480e-b751-be927f357d8f None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.654090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fbe61-afea-480e-b751-be927f357d8f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.654368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fbe61-afea-480e-b751-be927f357d8f None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.655913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 512/1046] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:10.662444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e16dd2-9e09-4bc0-8098-bd63d2dede49 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.662808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e16dd2-9e09-4bc0-8098-bd63d2dede49 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.663002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e16dd2-9e09-4bc0-8098-bd63d2dede49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.683733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-65e16dd2-9e09-4bc0-8098-bd63d2dede49 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:10.684901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-65e16dd2-9e09-4bc0-8098-bd63d2dede49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:10.717743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:10.758744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 535/1047] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:10.787342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca190318-6a80-4259-b77f-9a2007c10d3c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.787655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca190318-6a80-4259-b77f-9a2007c10d3c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.788012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca190318-6a80-4259-b77f-9a2007c10d3c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.791266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 513/1048] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:10.795073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.795478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.795764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.816335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:10.817576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:10.898100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 536/1049] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:11.041578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca190318-6a80-4259-b77f-9a2007c10d3c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.151875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca190318-6a80-4259-b77f-9a2007c10d3c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.168793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.204836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.205569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.205569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.205569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.218179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:11.223088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:11.224074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:11.225128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 514/1050] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:11.230898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.253888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.295829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.304036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707f775-0df4-454c-92b6-a3178d470e56 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.344178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.349775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.349775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.349775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.349775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.350999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:11.361071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:11.367353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:11.368378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 537/1051] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:11.369384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.377941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabb6a4-2874-4925-b2f1-0a4e4c61bf16 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.413669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.414273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.414548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.414815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.415508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:11.415814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.417714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.425875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 515/1052] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:11.432076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.486574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d73e7-5128-4a90-876e-4841c632d5d2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.522837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.524260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.525037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.525379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.525379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.526018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] RBAC: Adding query filter params (name=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:11.526326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.527793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.538255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 538/1053] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:11.539136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.544997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a9da-9d35-4e3c-baa1-36dbaa2bc8a9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.570700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.584428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.585072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.585359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.585635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] PATH_INFO: `/v3/users/2af50a501ad743d0add7716fccc2ef3a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.600314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] RBAC: Authorizing `identity:get_user(user_id=2af50a501ad743d0add7716fccc2ef3a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.602210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.609311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 516/1054] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/users/2af50a501ad743d0add7716fccc2ef3a => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:11.610153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.655125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899b4cd-44f1-45c0-960c-47df87baa96d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.690106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.690731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.690991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.691300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.691927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] RBAC: Adding query filter params (type=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:11.692235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.693607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.698751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 539/1055] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:11.704170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.718796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce4981-7fa5-4b7b-add3-e898ca38881c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.754293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.754906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.755185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.755484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.766859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.794294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.807558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61df66-1550-4722-b0fa-a17e984f2fff None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.828812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.830221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.842388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.843018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.843335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.843571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.844239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.845637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.871237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 540/1056] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:46:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:11.956211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 517/1057] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:11.961962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.084579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cff9d-3e5b-419c-b41a-f8e26cbaba99 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.119902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb321e58-43d4-4153-a680-5a4064d6f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.120645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb321e58-43d4-4153-a680-5a4064d6f87f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.121233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb321e58-43d4-4153-a680-5a4064d6f87f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.121739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb321e58-43d4-4153-a680-5a4064d6f87f None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.123540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb321e58-43d4-4153-a680-5a4064d6f87f None admin] RBAC: Authorizing `identity:get_project(project_id=42c3469ddd8d4bef8674c42ed4c528c1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.125007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb321e58-43d4-4153-a680-5a4064d6f87f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.126913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 541/1058] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.133071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.176944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.237067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e319-8f7f-4cd9-b617-3f5d76ae07a5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.271519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.272125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.272400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.272666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.273388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1, user.id=2af50a501ad743d0add7716fccc2ef3a) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.273687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.275107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208da38-a481-4205-83c5-af2a3032235d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.281556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 518/1059] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/role_assignments?user.id=2af50a501ad743d0add7716fccc2ef3a&scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.849696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504b415-a2fb-45c3-80b6-e462322cf55b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.850227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504b415-a2fb-45c3-80b6-e462322cf55b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.850227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504b415-a2fb-45c3-80b6-e462322cf55b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.851825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 542/1060] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:12.856931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.857462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.857462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.876004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:12.877165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:12.955408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 519/1061] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:13.219914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504b415-a2fb-45c3-80b6-e462322cf55b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.232890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.329378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504b415-a2fb-45c3-80b6-e462322cf55b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.362875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.363808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.364289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.364764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.376811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:13.381948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:13.383061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:13.384388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:13.386937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 543/1062] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:13.393002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.426414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.503359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ca5cc-0111-4f03-8972-4ec1ea68c123 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.538453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.539059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.539368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.539637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.540284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:13.540584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.542465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.555261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 520/1063] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:13.557465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.633908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.671867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703818-b036-4aa5-9ee4-de99a16c30d4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.707025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.707702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.707981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.708254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.722065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.723628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:13.740635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:13.742007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:13.742963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:13.742963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:13.742963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:13.743732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 544/1064] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:13.748168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.845216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.852096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd899d-66ec-4d32-8ef5-b3f9aae1452b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.889570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.890275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.890536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.890717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.891388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:13.891695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.893071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dff87c-ed08-476f-963d-5b93fc7d9a84 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.906412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 521/1065] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:13.911859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.026260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51c028-cd17-44eb-9508-8b905412295d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.048168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.063670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.064311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.064588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.064860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.073133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:14.076080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:46:14.087735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:14.088745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:14.089783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:14.091501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 545/1066] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:14.092675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5345813-a27a-4c6e-b35b-7f47b44dae86 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.092959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5345813-a27a-4c6e-b35b-7f47b44dae86 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.093234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5345813-a27a-4c6e-b35b-7f47b44dae86 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.094333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.094811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 522/1067] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:14.099989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.100338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.100678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.119154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:14.121051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:14.188765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 523/1068] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:14.211435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bd792-9916-4ec8-add2-e2c548d3d828 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.248490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.249411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.249862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.250339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.251253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:14.251713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:14.253313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:14.271724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 546/1069] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:14.277356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.377711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.395898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f3308-0d10-41be-90ea-bfad23e16e32 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.434019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.434653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.434929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.435224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.445691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:14.447137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:14.456240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a239f392-b27f-4f94-bcf1-08d4fa8548fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:14.463443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:14.464505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:14.465516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:14.465516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:14.465516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:14.465516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 524/1070] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:14.466239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fe1f6-d284-4926-b04f-d234c31fda69 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.466521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fe1f6-d284-4926-b04f-d234c31fda69 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.466794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fe1f6-d284-4926-b04f-d234c31fda69 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.468547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 525/1071] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:14.472588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fe1f6-d284-4926-b04f-d234c31fda69 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.504829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.573433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66122-73c0-479d-b8db-091fc96463c5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.583557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fe1f6-d284-4926-b04f-d234c31fda69 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.610999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.611768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.612050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.612325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.626035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.626643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.626923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.627192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.627865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:14.628170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:14.629536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e1b44-68c2-485f-8056-92654c399998 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:14.636500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.643546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 526/1072] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:14.644771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.645083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.645339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.664015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:14.665223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:14.688391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:14.689848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:14.731626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.735411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 527/1073] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:14.736674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.803358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 547/1074] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:46:14.805787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.847308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42ec35-3c86-47ab-8fb7-b9ffb328e12b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.881310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.892370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.893271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.893271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.893695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.902759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:14.904247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:46:14.917979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:14.919141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:14.920280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:14.922062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 528/1075] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:14.928539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:14.940798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab18ca1-3259-4292-9560-cb61572a562d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:14.984999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:14.985661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:14.985942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:14.986213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:14.993134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:14.997667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.000191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:15.015763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:15.016955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:15.018054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:15.018054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:15.018054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:15.018501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 548/1076] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:15.019366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.042366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126f18f-3cb6-42e7-829a-47496ace5a31 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.076250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.077224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.077224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.077449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.078032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:15.078332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.079809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.091607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 529/1077] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:15.092456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.128619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278cf9-5721-4694-9920-e8f7ccbe3985 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.142123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.163135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.163817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.164091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.164361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.175139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:15.180392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:15.183758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 549/1078] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:15.184621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.198723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73002140-a8f6-4265-a59b-e66a69d490b2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.232281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.232897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.233171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.233440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.234074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:15.234374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.235790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.237467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.249383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 530/1079] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:15.250216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.294304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e3b11-4f4d-44ed-aea2-80442a3579f2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.332891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.333565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.333846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.334122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/ff1a6a40166e4f51a14cca9cd510e680/roles/aebafcc9f64b42558dde9415d00cdedc` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.336410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=ff1a6a40166e4f51a14cca9cd510e680, role_id=aebafcc9f64b42558dde9415d00cdedc)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.337891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.353244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 550/1080] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:15 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/ff1a6a40166e4f51a14cca9cd510e680/roles/aebafcc9f64b42558dde9415d00cdedc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:15.382350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0542021-8523-4e85-b245-329712e8c54a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.395545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.427421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.428452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.428894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.429175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.430594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:15.430594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.431791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.443435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 531/1081] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:15.449705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.572031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd158fe-e6a2-43f3-b5c3-9a6e04ede915 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.584767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.613293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.613895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.614173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.614447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] PATH_INFO: `/v3/users/2af50a501ad743d0add7716fccc2ef3a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.615667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] RBAC: Authorizing `identity:get_user(user_id=2af50a501ad743d0add7716fccc2ef3a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.617636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.619266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 551/1082] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/users/2af50a501ad743d0add7716fccc2ef3a => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:15.625558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.723075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088264a-e157-440e-9fce-2fcee3192c36 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.756695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981fff8-5f6d-4291-946c-cab9a9ab1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.757313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981fff8-5f6d-4291-946c-cab9a9ab1594 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.757587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981fff8-5f6d-4291-946c-cab9a9ab1594 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.757853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981fff8-5f6d-4291-946c-cab9a9ab1594 None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.759221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981fff8-5f6d-4291-946c-cab9a9ab1594 None admin] RBAC: Authorizing `identity:get_project(project_id=42c3469ddd8d4bef8674c42ed4c528c1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.760667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981fff8-5f6d-4291-946c-cab9a9ab1594 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.762269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 532/1083] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:15.768243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.869397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81d861-f958-4acd-a85c-1a2303f596bb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.906766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38c513-c133-4d92-a680-16c495f92ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.907941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c513-c133-4d92-a680-16c495f92ac1 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.908231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c513-c133-4d92-a680-16c495f92ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.908505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38c513-c133-4d92-a680-16c495f92ac1 None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1/users/2af50a501ad743d0add7716fccc2ef3a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.910499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c513-c133-4d92-a680-16c495f92ac1 None admin] RBAC: Authorizing `identity:create_grant(project_id=42c3469ddd8d4bef8674c42ed4c528c1, user_id=2af50a501ad743d0add7716fccc2ef3a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.913010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38c513-c133-4d92-a680-16c495f92ac1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.927672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 552/1084] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:15 2022] PUT /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1/users/2af50a501ad743d0add7716fccc2ef3a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:17.552068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1382-5a71-41d7-bcde-6d7b394f3307 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.552631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1382-5a71-41d7-bcde-6d7b394f3307 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.552631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b1382-5a71-41d7-bcde-6d7b394f3307 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.553952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 533/1085] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:17.559397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f250-e119-4030-bfaf-c75be67f89a4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.559804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f250-e119-4030-bfaf-c75be67f89a4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.560084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53f250-e119-4030-bfaf-c75be67f89a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.578592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8c53f250-e119-4030-bfaf-c75be67f89a4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:17.579870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8c53f250-e119-4030-bfaf-c75be67f89a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:17.609448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:17.650244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 553/1086] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:17.922049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b1382-5a71-41d7-bcde-6d7b394f3307 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:17.967148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.033500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b1382-5a71-41d7-bcde-6d7b394f3307 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.065326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11c6c-1a6e-435a-94f7-493cdfe04e7b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.065615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11c6c-1a6e-435a-94f7-493cdfe04e7b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.065888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11c6c-1a6e-435a-94f7-493cdfe04e7b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.067259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 554/1087] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:18.067594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.068309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.068537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.068825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.072163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.072437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.072709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.079406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:18.084249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:18.085284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:18.086127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 534/1088] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Sep 22 14:46:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:18.091772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.091892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:18.093067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:18.170491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 555/1089] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:18.189769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.194029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3855d-6557-430a-b29b-f52a8fa8a7dc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.229608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.230201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.230473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.230740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.231531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:18.231716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.233071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.240706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 535/1090] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:18.246605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.341353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.345950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae4bf7-475f-40e6-abd1-87edc65520ac None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.382392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.383019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.383577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.383843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.394784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.396265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:18.411144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:18.412276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:18.413292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:18.413292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:18.413292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:18.413292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 556/1091] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:18.418207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.437408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.525162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40a88e-554a-4b72-8803-06eee4c78acf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.542997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c4712-a09c-46e1-9013-1c880b0563a3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.558298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.558896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.559172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.559473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.560099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:18.560399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.561757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.573743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 536/1092] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:18.577522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.578241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.578574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.578854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.579334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.590430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:18.595299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:18.596357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:18.597198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 557/1093] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:18.602192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.605166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.614461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.685722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91734d-be69-4f6a-84a6-f2e5be9acaa7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.702650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268fe58-908f-476b-9d06-e4c2e23c51d0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.719984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.720604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.720885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.721222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.728246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.730170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.735898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.736492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.736771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.737039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.737691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:18.737994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.739388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:46:18.741578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:18.742696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:18.744013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:18.747528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 537/1094] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:18.748477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.748611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 558/1095] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:18.754315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.822332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.829721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.850161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5145-8ad1-47ec-aa00-303ad3b04496 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.853913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8819d1-916f-4848-b897-dfd1b5c377ee None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.882290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.883126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.883748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.884031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.884830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:18.885266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.887313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.888717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.889356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.889635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.889913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] PATH_INFO: `/v3/users/2af50a501ad743d0add7716fccc2ef3a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.891064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] RBAC: Authorizing `identity:get_user(user_id=2af50a501ad743d0add7716fccc2ef3a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.892587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.894484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 559/1096] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/users/2af50a501ad743d0add7716fccc2ef3a => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:18.901164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.907631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 538/1097] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:18.913293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.002535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d72fc8-5275-4b51-b22f-0afb46877a87 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.012304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf877562-b26a-425c-9f4d-4643164675af None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.037093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.037692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.037971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.038238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.046376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.046971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.047272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.047543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.048666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.055884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.106918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.108411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.114533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.115998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.207980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 560/1098] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:46:19.213837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.218289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 539/1099] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:19.224285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.254580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.265331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.318721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfbb1d-f353-4bc9-a150-56ddb8056ab8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.328828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55594c02-152d-4bcf-bfec-fa0c52217fee None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.354851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.355546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.355824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.356095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.357234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] RBAC: Authorizing `identity:get_project(project_id=42c3469ddd8d4bef8674c42ed4c528c1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.358813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.360565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 561/1100] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:19.363260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.363885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.364164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.364438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.366542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.375374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.376823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:19.393212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:19.394255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:19.395145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:19.395145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:19.395145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:19.395145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 540/1101] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:19.399981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.469762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbae3f5-5321-40e5-88ac-411287e6d97a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.482007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.493768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.504187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399509c-b32b-43fa-80de-a6e05ceae605 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.517841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.518381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.518660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.518934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.519718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1, user.id=2af50a501ad743d0add7716fccc2ef3a) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:19.520028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.521691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.528012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 562/1102] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/role_assignments?user.id=2af50a501ad743d0add7716fccc2ef3a&scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:19.545533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.546166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.546445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.546717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.547419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:19.547881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.549202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.564183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 541/1103] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:19.570218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.703221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d148fe8-8efe-43b6-b4b4-e3b9008ef39a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.741660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.742281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.742560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.742825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.750012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.751479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:46:19.762857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:19.763809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:19.764714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 563/1104] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:19.769958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.871685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc755b97-c400-4f3f-b136-f7d30448707c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.905738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.906341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.906613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.906880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.907596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:19.907895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.909308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe66e5-4ab4-4c20-a47d-0418fd2de3fa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.922258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 542/1105] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:19.929727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.961102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:20.035369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e3a6-e2e8-4e0d-96d3-656eb7e37ede None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:20.074757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:20.075387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:20.075663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:20.075930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:20.076641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] RBAC: Adding query filter params (role.id=aebafcc9f64b42558dde9415d00cdedc, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=ff1a6a40166e4f51a14cca9cd510e680) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:20.076944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:20.078367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7841f09-2349-4c6c-a575-d0935762eeb3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:20.084127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 564/1106] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/role_assignments?user.id=ff1a6a40166e4f51a14cca9cd510e680&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=aebafcc9f64b42558dde9415d00cdedc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:21.748493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdd186-9eb4-4ab6-8d78-2e044df581ad None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:21.749023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdd186-9eb4-4ab6-8d78-2e044df581ad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:21.749023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdd186-9eb4-4ab6-8d78-2e044df581ad None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:21.750310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 543/1107] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:21.755503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:21.755907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:21.756114 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:21.774781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:21.775995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:21.840829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 565/1108] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:22.110696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdd186-9eb4-4ab6-8d78-2e044df581ad None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:22.126075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:22.219953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdd186-9eb4-4ab6-8d78-2e044df581ad None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:22.254497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5ee0e-bf5d-4c82-b29d-3c773fd944a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:22.255129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ee0e-bf5d-4c82-b29d-3c773fd944a2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:22.255437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ee0e-bf5d-4c82-b29d-3c773fd944a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:22.255711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5ee0e-bf5d-4c82-b29d-3c773fd944a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:22.256827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5ee0e-bf5d-4c82-b29d-3c773fd944a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:22.258515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5ee0e-bf5d-4c82-b29d-3c773fd944a2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:22.260118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 544/1109] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:22.266547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:22.293096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:22.365332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a325b-baf6-4369-8a99-5191132f1b1b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:22.399625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64d7a4-a692-4df6-9d72-5cea948af81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:22.400235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64d7a4-a692-4df6-9d72-5cea948af81e None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:22.400514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64d7a4-a692-4df6-9d72-5cea948af81e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:22.400812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64d7a4-a692-4df6-9d72-5cea948af81e None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:22.401966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64d7a4-a692-4df6-9d72-5cea948af81e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:22.403419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64d7a4-a692-4df6-9d72-5cea948af81e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:22.425045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 566/1110] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:46:22 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:24.651899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5236a56-a251-4320-b169-84a056afb566 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:24.652431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5236a56-a251-4320-b169-84a056afb566 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:24.652518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5236a56-a251-4320-b169-84a056afb566 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:24.653999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 545/1111] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:24.668852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:24.669682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:24.669682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:24.689732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:24.690932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:24.763000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 567/1112] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:25.038093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5236a56-a251-4320-b169-84a056afb566 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:25.117115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:25.145945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5236a56-a251-4320-b169-84a056afb566 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:25.185684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:25.186321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.186597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.186872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] PATH_INFO: `/v3/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.188304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] RBAC: Authorizing `identity:get_role(role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:25.189674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:25.191009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 546/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/roles/e1b5b7a11157451ab80ef767124856f0 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:25.197494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:25.213411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:25.306305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafbe58-adef-4674-b54b-3612e6396cdb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:25.353065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:25.353340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.353498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.353771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] PATH_INFO: `/v3/users/0805f46eaf814851aaaea9812c414d58` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.368320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=0805f46eaf814851aaaea9812c414d58)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:25.370313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:25.372175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 568/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/users/0805f46eaf814851aaaea9812c414d58 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:25.378231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:25.492041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91c671-b594-48b8-9ecc-d6b5a065ad6c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:25.528112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:25.528723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.529002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.529276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.544494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:25.596223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:25.597595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:25.700130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 547/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:25.706240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:25.775340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:25.807632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bd61a-5a9d-4e6d-9eac-822b45819eb0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:25.841923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca0429-38a1-4b9c-a4f5-408cdb5641ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:25.842547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca0429-38a1-4b9c-a4f5-408cdb5641ab None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.842820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca0429-38a1-4b9c-a4f5-408cdb5641ab None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.843092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca0429-38a1-4b9c-a4f5-408cdb5641ab None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.844224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca0429-38a1-4b9c-a4f5-408cdb5641ab None admin] RBAC: Authorizing `identity:get_project(project_id=42c3469ddd8d4bef8674c42ed4c528c1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:25.845685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca0429-38a1-4b9c-a4f5-408cdb5641ab None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:25.847407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 569/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:25.853801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:25.901456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:25.956819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babab4a5-65ac-4281-bfb0-04547bee050e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:25.991229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:25.991854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.992135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.992406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.993135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] RBAC: Adding query filter params (role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1, user.id=0805f46eaf814851aaaea9812c414d58) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:25.993446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:25.995444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33096953-0917-40fb-a03f-c1e4ae04935f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:26.001208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 548/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/role_assignments?user.id=0805f46eaf814851aaaea9812c414d58&scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:28.170610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9f212-4f1e-4c67-a86a-804bd8625e85 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.171180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9f212-4f1e-4c67-a86a-804bd8625e85 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.171180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9f212-4f1e-4c67-a86a-804bd8625e85 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.172530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 570/1118] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:28.177952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.178408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.178502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.196298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:28.197581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:28.263083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 549/1119] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:28.536429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9f212-4f1e-4c67-a86a-804bd8625e85 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:28.637410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9f212-4f1e-4c67-a86a-804bd8625e85 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:28.670776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:28.671431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.671708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.671981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] PATH_INFO: `/v3/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.673362 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] RBAC: Authorizing `identity:get_role(role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:28.674760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:28.676124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 571/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:28 2022] GET /identity/v3/roles/e1b5b7a11157451ab80ef767124856f0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:28.682550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:28.722554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:28.781679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8d70-759e-4a39-9842-cd1fd807bf6f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:28.816365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:28.817018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.817266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.817535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] PATH_INFO: `/v3/users/0805f46eaf814851aaaea9812c414d58` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.818656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] RBAC: Authorizing `identity:get_user(user_id=0805f46eaf814851aaaea9812c414d58)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:28.820123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:28.821703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 550/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:28 2022] GET /identity/v3/users/0805f46eaf814851aaaea9812c414d58 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:28.827662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:28.836836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:28.930116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367eaa-a5e7-42e1-94bb-a81a70afb971 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:28.962502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb485e-8a94-44c9-8868-7af9500c6cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:28.963661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb485e-8a94-44c9-8868-7af9500c6cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.963943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb485e-8a94-44c9-8868-7af9500c6cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.964214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb485e-8a94-44c9-8868-7af9500c6cb8 None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.965316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb485e-8a94-44c9-8868-7af9500c6cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=42c3469ddd8d4bef8674c42ed4c528c1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:28.966747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb485e-8a94-44c9-8868-7af9500c6cb8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:28.968416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 572/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:28 2022] GET /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:28.974781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:29.074380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462babf-1c7b-414a-8e48-05a7089c4219 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:29.083805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:29.120041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df104be-5998-4858-ae74-637a4618c260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:29.120735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df104be-5998-4858-ae74-637a4618c260 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:29.121011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df104be-5998-4858-ae74-637a4618c260 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:29.121285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df104be-5998-4858-ae74-637a4618c260 None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1/users/0805f46eaf814851aaaea9812c414d58/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:29.123732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df104be-5998-4858-ae74-637a4618c260 None admin] RBAC: Authorizing `identity:create_grant(project_id=42c3469ddd8d4bef8674c42ed4c528c1, user_id=0805f46eaf814851aaaea9812c414d58, role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:29.125244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df104be-5998-4858-ae74-637a4618c260 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:29.147148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 551/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:28 2022] PUT /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1/users/0805f46eaf814851aaaea9812c414d58/roles/e1b5b7a11157451ab80ef767124856f0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:31.345998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59d44a-4cdb-498f-8382-7a02d74b838b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:31.346611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59d44a-4cdb-498f-8382-7a02d74b838b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:31.346611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59d44a-4cdb-498f-8382-7a02d74b838b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:31.347901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 573/1124] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:31.353116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:31.353491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:31.353760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:31.374889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:31.378075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:31.445018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 552/1125] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:31.723234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59d44a-4cdb-498f-8382-7a02d74b838b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:31.804395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:31.832897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59d44a-4cdb-498f-8382-7a02d74b838b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:31.866871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:31.867530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:31.867805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:31.868073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] PATH_INFO: `/v3/roles/e1b5b7a11157451ab80ef767124856f0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:31.870002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] RBAC: Authorizing `identity:get_role(role_id=e1b5b7a11157451ab80ef767124856f0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:31.871401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:31.872732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 574/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:31 2022] GET /identity/v3/roles/e1b5b7a11157451ab80ef767124856f0 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:31.878754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:31.976153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:31.980577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451405d-1994-4af9-baec-858b14a733e4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.017030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.017647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.017925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.018197 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] PATH_INFO: `/v3/users/0805f46eaf814851aaaea9812c414d58` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.019328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] RBAC: Authorizing `identity:get_user(user_id=0805f46eaf814851aaaea9812c414d58)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.020751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.022304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 553/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:31 2022] GET /identity/v3/users/0805f46eaf814851aaaea9812c414d58 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:32.028257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:32.127154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0a75-213f-4cef-b93b-36b0c480e622 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.161675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.162274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.162552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.162823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.174818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:32.232769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.234262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.334834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 575/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:32.340950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:32.436634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599cb2-191f-46f5-88a4-cbc06ca63111 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.471403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737ee84-c499-4448-9645-7d143cd3ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.472153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ee84-c499-4448-9645-7d143cd3ef35 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.472319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ee84-c499-4448-9645-7d143cd3ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.472591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737ee84-c499-4448-9645-7d143cd3ef35 None admin] PATH_INFO: `/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.474192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ee84-c499-4448-9645-7d143cd3ef35 None admin] RBAC: Authorizing `identity:get_project(project_id=42c3469ddd8d4bef8674c42ed4c528c1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.475401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737ee84-c499-4448-9645-7d143cd3ef35 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.477104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 554/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/projects/42c3469ddd8d4bef8674c42ed4c528c1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:32.484063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:32.528755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:32.585288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e69d-c267-4eda-9565-78ac4e729bfb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.618793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.619444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.619718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.619986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.620689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] RBAC: Adding query filter params (role.id=e1b5b7a11157451ab80ef767124856f0, scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1, user.id=0805f46eaf814851aaaea9812c414d58) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:32.620992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.622399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840129b-7417-4689-81d9-3718a72e8db5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.628008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 576/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/role_assignments?user.id=0805f46eaf814851aaaea9812c414d58&scope.project.id=42c3469ddd8d4bef8674c42ed4c528c1&role.id=e1b5b7a11157451ab80ef767124856f0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:34.822660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5331a-3e00-4cbb-8b3e-e4eae8b8ee46 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:34.823252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5331a-3e00-4cbb-8b3e-e4eae8b8ee46 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:34.823252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5331a-3e00-4cbb-8b3e-e4eae8b8ee46 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:34.824556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 555/1131] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:34.829862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:34.830289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:34.830409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:34.848885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:34.850069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:34.921438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 577/1132] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:35.191013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5331a-3e00-4cbb-8b3e-e4eae8b8ee46 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:35.225487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:35.291670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5331a-3e00-4cbb-8b3e-e4eae8b8ee46 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:35.324434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df34a99-b2b3-4251-954e-53d0165ab203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:35.325057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df34a99-b2b3-4251-954e-53d0165ab203 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:35.325336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df34a99-b2b3-4251-954e-53d0165ab203 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:35.325607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df34a99-b2b3-4251-954e-53d0165ab203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:35.326716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df34a99-b2b3-4251-954e-53d0165ab203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:35.328171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df34a99-b2b3-4251-954e-53d0165ab203 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:35.329719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 556/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:35.335607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:35.370099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:35.438117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbff63c-7e32-4c9e-8571-9a6a4b70f6d0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:35.477020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3c2e3-30f8-442d-adc9-46b9c04b14c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:35.477643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3c2e3-30f8-442d-adc9-46b9c04b14c0 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:35.477920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3c2e3-30f8-442d-adc9-46b9c04b14c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:35.478216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3c2e3-30f8-442d-adc9-46b9c04b14c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:35.478879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3c2e3-30f8-442d-adc9-46b9c04b14c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:35.480467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3c2e3-30f8-442d-adc9-46b9c04b14c0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:35.508682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 578/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:35 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:37.659937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484caf1-8fea-4d37-879c-e90cded2755f None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:37.660460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484caf1-8fea-4d37-879c-e90cded2755f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:37.660460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484caf1-8fea-4d37-879c-e90cded2755f None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:37.661805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 557/1135] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:37.667058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:37.667528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:37.667623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:37.686094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:37.687297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:37.756524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 579/1136] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:38.033319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484caf1-8fea-4d37-879c-e90cded2755f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:38.132213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484caf1-8fea-4d37-879c-e90cded2755f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:38.133965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:38.166401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0525d3-f6a5-4c38-8a50-283a1dbd5791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:38.167021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0525d3-f6a5-4c38-8a50-283a1dbd5791 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:38.167329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0525d3-f6a5-4c38-8a50-283a1dbd5791 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:38.167604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0525d3-f6a5-4c38-8a50-283a1dbd5791 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:38.168701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0525d3-f6a5-4c38-8a50-283a1dbd5791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:38.170121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0525d3-f6a5-4c38-8a50-283a1dbd5791 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:38.171709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 558/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:38.177493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:38.195884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:38.276598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6661225-41ab-48c8-b48d-d8fce0aa477e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:38.310778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d813114-b375-4e1d-a5f4-307b608adcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:38.311411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d813114-b375-4e1d-a5f4-307b608adcbe None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:38.311694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d813114-b375-4e1d-a5f4-307b608adcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:38.311963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d813114-b375-4e1d-a5f4-307b608adcbe None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:38.312605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d813114-b375-4e1d-a5f4-307b608adcbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:38.314080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d813114-b375-4e1d-a5f4-307b608adcbe None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:38.334540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 580/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:46:38 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:40.529407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6b8c5-553f-4dc1-acf5-e53b63d365a4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:40.529954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6b8c5-553f-4dc1-acf5-e53b63d365a4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:40.529954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6b8c5-553f-4dc1-acf5-e53b63d365a4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:40.531396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 559/1139] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:40.536653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:40.537061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:40.537201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:40.554929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:40.556158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:40.622189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 581/1140] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:40.903996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6b8c5-553f-4dc1-acf5-e53b63d365a4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.005206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6b8c5-553f-4dc1-acf5-e53b63d365a4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.040280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.040944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.041220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.041488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.051604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:41.056974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:41.058096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:41.059148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 560/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:41.064274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.067405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.164459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f72e8-3dd0-4006-a49c-765ec43d00f7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.204696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.204984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.205261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.206039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:41.206039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:41.207619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:41.215806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 582/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:41.221762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.253446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.323782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2c14-3037-459f-bcc1-3416c2e9caf1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.361969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.362595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.362875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.363150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] PATH_INFO: `/v3/users/351e4706733d46aa91ce5b0f2b6953a2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.378138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] RBAC: Authorizing `identity:get_user(user_id=351e4706733d46aa91ce5b0f2b6953a2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:41.379620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:41.381279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 561/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/users/351e4706733d46aa91ce5b0f2b6953a2 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:41.387618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.455624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.490442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771ce48-2c50-46a6-8feb-38b9beb3bc42 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.532322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.532928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.533206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.533478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.604942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:41.606391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:41.667757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.704025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 583/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:41.710065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.795043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.807475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34303-fe69-4635-9b7e-d23d2a82c560 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.840489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30890b1f-93a4-44d5-afd5-36a0656d83b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.841119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30890b1f-93a4-44d5-afd5-36a0656d83b3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.841398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30890b1f-93a4-44d5-afd5-36a0656d83b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.841669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30890b1f-93a4-44d5-afd5-36a0656d83b3 None admin] PATH_INFO: `/v3/projects/95c04b475f8b4beebc0fed37990d9933` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.842782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30890b1f-93a4-44d5-afd5-36a0656d83b3 None admin] RBAC: Authorizing `identity:get_project(project_id=95c04b475f8b4beebc0fed37990d9933)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:41.844243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30890b1f-93a4-44d5-afd5-36a0656d83b3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:41.845850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 562/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/projects/95c04b475f8b4beebc0fed37990d9933 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:41.852156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.964193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c695de7-c81d-40bd-a71d-b7b1a22ab464 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:42.000594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:42.001201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:42.001482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:42.001754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:42.002474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=95c04b475f8b4beebc0fed37990d9933, user.id=351e4706733d46aa91ce5b0f2b6953a2) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:42.002781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:42.004241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a673d-081f-470c-9b0b-e7a6b22e6bb3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:42.009890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 584/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/role_assignments?user.id=351e4706733d46aa91ce5b0f2b6953a2&scope.project.id=95c04b475f8b4beebc0fed37990d9933&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:44.133435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb989813-a5c4-489a-92fc-04a1dee6859a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.134020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb989813-a5c4-489a-92fc-04a1dee6859a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.134020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb989813-a5c4-489a-92fc-04a1dee6859a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.135372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 563/1147] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:44.140637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.141031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.141187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.159436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:44.160710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:44.183865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:44.226776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 585/1148] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:44.501082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb989813-a5c4-489a-92fc-04a1dee6859a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:44.602577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb989813-a5c4-489a-92fc-04a1dee6859a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:44.637396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:44.638021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.638300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.638568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.648290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:44.653072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:44.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:44.655527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 564/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:44.660259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:44.757490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ec5f4-9398-425d-ad40-48ee7e4fdc9b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:44.791399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:44.792152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.792429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.792698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.793359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:44.793660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:44.795029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:44.803070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 586/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:44.809566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:44.828484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:44.909976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895f0ec-bef7-4e3d-a0e8-f811fc2127b7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:44.943390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:44.944014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.944294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.944566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] PATH_INFO: `/v3/users/351e4706733d46aa91ce5b0f2b6953a2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.945682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] RBAC: Authorizing `identity:get_user(user_id=351e4706733d46aa91ce5b0f2b6953a2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:44.947759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:44.949334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 565/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:44 2022] GET /identity/v3/users/351e4706733d46aa91ce5b0f2b6953a2 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:44.955142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:44.979546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:45.052674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c9e24-b2c1-468a-8ee8-dbbb277e9d7b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.089012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d1057-0555-4bd5-a559-2687e17d55fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.089635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1057-0555-4bd5-a559-2687e17d55fe None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.089918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1057-0555-4bd5-a559-2687e17d55fe None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.090191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d1057-0555-4bd5-a559-2687e17d55fe None admin] PATH_INFO: `/v3/projects/95c04b475f8b4beebc0fed37990d9933` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.091324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d1057-0555-4bd5-a559-2687e17d55fe None admin] RBAC: Authorizing `identity:get_project(project_id=95c04b475f8b4beebc0fed37990d9933)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:45.092752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d1057-0555-4bd5-a559-2687e17d55fe None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:45.094385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 587/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:44 2022] GET /identity/v3/projects/95c04b475f8b4beebc0fed37990d9933 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:45.100376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:45.181245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:45.196822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deadc7f-f5e0-443c-b4c2-0ae42d8b09d6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.229392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d130d7-45c9-4d1f-bcbc-7cfa0fdc6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.230034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d130d7-45c9-4d1f-bcbc-7cfa0fdc6a4c None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.230311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d130d7-45c9-4d1f-bcbc-7cfa0fdc6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.230580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d130d7-45c9-4d1f-bcbc-7cfa0fdc6a4c None admin] PATH_INFO: `/v3/projects/95c04b475f8b4beebc0fed37990d9933/users/351e4706733d46aa91ce5b0f2b6953a2/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.232475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d130d7-45c9-4d1f-bcbc-7cfa0fdc6a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=95c04b475f8b4beebc0fed37990d9933, user_id=351e4706733d46aa91ce5b0f2b6953a2, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:45.233899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d130d7-45c9-4d1f-bcbc-7cfa0fdc6a4c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:45.248228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 566/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:45 2022] PUT /identity/v3/projects/95c04b475f8b4beebc0fed37990d9933/users/351e4706733d46aa91ce5b0f2b6953a2/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:47.389950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbafff-10e3-452d-a3d2-157dac4611dd None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:47.390516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbafff-10e3-452d-a3d2-157dac4611dd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:47.390516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbafff-10e3-452d-a3d2-157dac4611dd None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:47.391916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 588/1154] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:47.397188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:47.397632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:47.397727 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:47.416062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:47.417232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:47.488023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 567/1155] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:47.757771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbafff-10e3-452d-a3d2-157dac4611dd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:47.848345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:47.859072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbafff-10e3-452d-a3d2-157dac4611dd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:47.893479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:47.894113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:47.894395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:47.894669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:47.905515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:47.910369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:47.911494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:46:47.912747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 589/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:47.917680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:47.989977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.014957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba9951-4ef0-4e75-ba2a-0ea173cc4658 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.047098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.047722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.047998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.048268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.048923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:48.049217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.050575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.058654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 568/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:48.064612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.163661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b5c2-fbd8-4ab7-8bb9-11204d85f183 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.199152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.199496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.199798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.200095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] PATH_INFO: `/v3/users/351e4706733d46aa91ce5b0f2b6953a2` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.201221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] RBAC: Authorizing `identity:get_user(user_id=351e4706733d46aa91ce5b0f2b6953a2)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.202662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.204289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 590/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/users/351e4706733d46aa91ce5b0f2b6953a2 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:48.210135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.308068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662dd6d-0ce2-499a-8320-bfac653b1f0b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.340514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.341115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.341392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.341666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.345006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.405752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.407194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.515019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 569/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:48.521220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.542129 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.620811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ea51d-c86b-4bad-af69-0ab98abcf7f2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.653982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57ae52-b0cf-4dbd-b835-e3d43b912037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.654608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57ae52-b0cf-4dbd-b835-e3d43b912037 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.654887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57ae52-b0cf-4dbd-b835-e3d43b912037 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.655163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57ae52-b0cf-4dbd-b835-e3d43b912037 None admin] PATH_INFO: `/v3/projects/95c04b475f8b4beebc0fed37990d9933` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.656393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57ae52-b0cf-4dbd-b835-e3d43b912037 None admin] RBAC: Authorizing `identity:get_project(project_id=95c04b475f8b4beebc0fed37990d9933)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.657820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57ae52-b0cf-4dbd-b835-e3d43b912037 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.659760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 591/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/projects/95c04b475f8b4beebc0fed37990d9933 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:48.666623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.699405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.762137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9423-73f9-4e38-bc1f-8bafdd59dd7d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.794957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.795575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.795853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.796138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.796831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=95c04b475f8b4beebc0fed37990d9933, user.id=351e4706733d46aa91ce5b0f2b6953a2) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:48.797138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.798560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc90c9-5217-4627-8307-c76a2020c97a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.804261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 570/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/role_assignments?user.id=351e4706733d46aa91ce5b0f2b6953a2&scope.project.id=95c04b475f8b4beebc0fed37990d9933&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:50.959317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb73d-1e39-4362-bf43-849266401433 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:50.959852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb73d-1e39-4362-bf43-849266401433 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:50.959852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb73d-1e39-4362-bf43-849266401433 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:50.961199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 592/1162] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:50.967086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:50.967545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:50.967676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:50.986195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:50.987420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:51.051883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 571/1163] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:51.316068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fb73d-1e39-4362-bf43-849266401433 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:51.403399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:51.419455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fb73d-1e39-4362-bf43-849266401433 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:51.451303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:51.451902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:51.452189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:51.452461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:51.518545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:51.520023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:51.600125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:51.615598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 593/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:51.621551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:51.650883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:51.718319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953b0f9-73e0-46bc-ae62-06ecd7eed03e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:51.751700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:51.752313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:51.752587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:51.752855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:51.759995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:51.761409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-966cfc67-c8da-420b-8fdc-f87a7166f4ef None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:51.772345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:51.773606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:46:51.774516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 572/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:46:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:51.781589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:51.877836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bf140-1f77-44d1-bce9-77fced3e66c8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:51.913511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:51.914107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:51.914385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:51.914656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:51.915318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:51.915651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:51.917059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63246e64-ebda-4665-97f5-94f492defcc7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:51.926780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 594/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:54.058960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085d61-8e27-474f-8ca0-1059625a942b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:54.059530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085d61-8e27-474f-8ca0-1059625a942b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:54.059530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085d61-8e27-474f-8ca0-1059625a942b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:54.060882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 573/1167] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:54.066014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704db1a7-c4e4-4dcb-a6eb-c94cc5fd3c86 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:54.066406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704db1a7-c4e4-4dcb-a6eb-c94cc5fd3c86 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:54.066568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704db1a7-c4e4-4dcb-a6eb-c94cc5fd3c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:54.084307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-704db1a7-c4e4-4dcb-a6eb-c94cc5fd3c86 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:54.085479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-704db1a7-c4e4-4dcb-a6eb-c94cc5fd3c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:54.131023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:54.151761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 595/1168] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:54.424945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0085d61-8e27-474f-8ca0-1059625a942b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:54.521381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:54.532375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0085d61-8e27-474f-8ca0-1059625a942b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:54.564942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7dd0b-dd2a-45a4-b76f-121e96d5eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:54.565542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7dd0b-dd2a-45a4-b76f-121e96d5eba9 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:54.565821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7dd0b-dd2a-45a4-b76f-121e96d5eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:54.566094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7dd0b-dd2a-45a4-b76f-121e96d5eba9 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:54.566761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7dd0b-dd2a-45a4-b76f-121e96d5eba9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:54.568154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7dd0b-dd2a-45a4-b76f-121e96d5eba9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:54.583078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 574/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Thu Sep 22 14:46:54 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:56.792711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01996340-b191-4129-a514-d57d52649286 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:56.793223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01996340-b191-4129-a514-d57d52649286 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:56.793316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01996340-b191-4129-a514-d57d52649286 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:56.794622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 596/1170] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:56.799904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:56.800356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:56.800464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:56.818830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:56.820060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:56.885070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 575/1171] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:57.164297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01996340-b191-4129-a514-d57d52649286 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:57.263743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01996340-b191-4129-a514-d57d52649286 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:57.297904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995408ce-f088-4abe-bb6e-c642fbd719f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:57.298526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995408ce-f088-4abe-bb6e-c642fbd719f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:57.298805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995408ce-f088-4abe-bb6e-c642fbd719f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:57.299077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995408ce-f088-4abe-bb6e-c642fbd719f4 None admin] PATH_INFO: `/v3/domains/290c28f82f0d48b696a1a3586e7e9bd4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:57.310563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995408ce-f088-4abe-bb6e-c642fbd719f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=290c28f82f0d48b696a1a3586e7e9bd4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:57.312350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995408ce-f088-4abe-bb6e-c642fbd719f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:57.313983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 597/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Thu Sep 22 14:46:57 2022] GET /identity/v3/domains/290c28f82f0d48b696a1a3586e7e9bd4 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:57.319935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:57.401302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:57.420771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17654e1-fbcc-485a-8b1c-43ec6752c98f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:57.452261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12815b-4205-4a57-b843-3350ccc42a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:57.452850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12815b-4205-4a57-b843-3350ccc42a56 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:57.453123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12815b-4205-4a57-b843-3350ccc42a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:57.453387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12815b-4205-4a57-b843-3350ccc42a56 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:57.454028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12815b-4205-4a57-b843-3350ccc42a56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:57.455471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12815b-4205-4a57-b843-3350ccc42a56 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:57.482649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 576/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:57 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:59.640123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f134bb-b330-4998-90db-c0c9348943ea None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:59.640674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f134bb-b330-4998-90db-c0c9348943ea None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:59.640674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f134bb-b330-4998-90db-c0c9348943ea None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:59.642006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 598/1174] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:59.647426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:59.648175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:59.648452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:59.667623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:59.668806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:59.735913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 577/1175] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:00.005427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f134bb-b330-4998-90db-c0c9348943ea None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:00.054836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:00.115782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f134bb-b330-4998-90db-c0c9348943ea None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:00.150456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b2b5a-3546-4352-a5db-b6cbe0502678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:00.151074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2b5a-3546-4352-a5db-b6cbe0502678 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:00.151387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2b5a-3546-4352-a5db-b6cbe0502678 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:00.151667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2b5a-3546-4352-a5db-b6cbe0502678 None admin] PATH_INFO: `/v3/domains/290c28f82f0d48b696a1a3586e7e9bd4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:00.152782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b2b5a-3546-4352-a5db-b6cbe0502678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=290c28f82f0d48b696a1a3586e7e9bd4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:00.154222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b2b5a-3546-4352-a5db-b6cbe0502678 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:00.155836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 599/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Thu Sep 22 14:47:00 2022] GET /identity/v3/domains/290c28f82f0d48b696a1a3586e7e9bd4 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:00.161724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:00.220171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:00.258130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249de409-b331-4eb9-8ca3-e6748bc6fce9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:00.292217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574fe10-1523-40bb-9259-1e382927ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:00.292805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574fe10-1523-40bb-9259-1e382927ec77 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:00.293078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574fe10-1523-40bb-9259-1e382927ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:00.293344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574fe10-1523-40bb-9259-1e382927ec77 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:00.293973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574fe10-1523-40bb-9259-1e382927ec77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:00.295414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574fe10-1523-40bb-9259-1e382927ec77 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:00.316302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 578/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:47:00 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:47:02.567035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942beb8a-2758-4c27-af45-d59dc5033359 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:02.567584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942beb8a-2758-4c27-af45-d59dc5033359 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:02.567697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942beb8a-2758-4c27-af45-d59dc5033359 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:02.568959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 600/1178] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:02.574215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:02.574587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:02.574755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:02.593230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:02.594414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:02.660471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 579/1179] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:02.931503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942beb8a-2758-4c27-af45-d59dc5033359 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.031366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942beb8a-2758-4c27-af45-d59dc5033359 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.040173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.064634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.065251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.065529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.065803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.076626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:47:03.081436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:47:03.082561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:47:03.083666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 601/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:47:03.089215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.141674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.191029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a3850-3541-45c6-b519-15f4d8fff8a5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.223921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.224523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.224797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.225063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.225711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:03.226011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:03.227395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:03.235748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 580/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:03.241654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.338404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039329f-d6ad-4a03-bce1-d307ec7e8c03 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.370553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.371161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.371465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.371732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] PATH_INFO: `/v3/users/834dddb604f14873b49b86724b8a2e1b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.384544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] RBAC: Authorizing `identity:get_user(user_id=834dddb604f14873b49b86724b8a2e1b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:03.385975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:03.387586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 602/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/users/834dddb604f14873b49b86724b8a2e1b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:03.393459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.494616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f9f7f-0bdf-45a9-8fd2-f2ba65c884e7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.505270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.531004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.531638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.531923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.532200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.601550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:03.603859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:03.702427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 581/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:47:03.708844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.736337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.806442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e7415-9bce-4fbb-852a-0353f84c0431 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.841678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faae55-1614-4dc1-9db7-8d253b512f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.842300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faae55-1614-4dc1-9db7-8d253b512f82 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.842579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faae55-1614-4dc1-9db7-8d253b512f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.842845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faae55-1614-4dc1-9db7-8d253b512f82 None admin] PATH_INFO: `/v3/projects/a175feac96244075a51ee95bfe5c58d9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.844003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95faae55-1614-4dc1-9db7-8d253b512f82 None admin] RBAC: Authorizing `identity:get_project(project_id=a175feac96244075a51ee95bfe5c58d9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:03.845426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95faae55-1614-4dc1-9db7-8d253b512f82 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:03.847052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 603/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/projects/a175feac96244075a51ee95bfe5c58d9 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:03.853318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.865553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.956806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abace35-3470-4ac3-b309-b5a33893aca6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.990205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.990725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.991007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.991312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.992064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=a175feac96244075a51ee95bfe5c58d9, user.id=834dddb604f14873b49b86724b8a2e1b) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:03.992371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:03.993808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46fb1a-f890-4ad0-8e43-43811ef7cf80 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:03.999103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 582/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/role_assignments?user.id=834dddb604f14873b49b86724b8a2e1b&scope.project.id=a175feac96244075a51ee95bfe5c58d9&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:06.186048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea228a-2208-48fc-a85b-e622bcd130ff None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:06.186587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea228a-2208-48fc-a85b-e622bcd130ff None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:06.186702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea228a-2208-48fc-a85b-e622bcd130ff None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:06.187996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 604/1186] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:06.193179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:06.193587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:06.193732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:06.213357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:06.214544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:06.278916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 583/1187] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:06.547437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea228a-2208-48fc-a85b-e622bcd130ff None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:06.649411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:06.657240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea228a-2208-48fc-a85b-e622bcd130ff None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:06.691954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:06.692583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:06.692865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:06.693138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:06.703701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:47:06.708539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:47:06.709705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:47:06.710591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 605/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:47:06.715902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:06.718958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:06.812453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbdaa2-a7df-40a1-b2fd-c48d7bd57dc6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:06.846348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:06.846940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:06.847229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:06.847510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:06.848156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:06.848451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:06.849804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:06.858017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 584/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:06.864122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:06.972770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e81234-d0ce-4920-b846-c052ed8b45c0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.017160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.017783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.018063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.018339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] PATH_INFO: `/v3/users/834dddb604f14873b49b86724b8a2e1b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.019528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] RBAC: Authorizing `identity:get_user(user_id=834dddb604f14873b49b86724b8a2e1b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.020960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.022551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 606/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:47:06 2022] GET /identity/v3/users/834dddb604f14873b49b86724b8a2e1b => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:47:07.028576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.094659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:07.128928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009cfd3-eae6-499a-bb5e-e14c2ce9060f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.162559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5a9f8-9dc3-4877-bd4b-356262727ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.163243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a9f8-9dc3-4877-bd4b-356262727ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.163532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a9f8-9dc3-4877-bd4b-356262727ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.163805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5a9f8-9dc3-4877-bd4b-356262727ce6 None admin] PATH_INFO: `/v3/projects/a175feac96244075a51ee95bfe5c58d9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.165544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5a9f8-9dc3-4877-bd4b-356262727ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=a175feac96244075a51ee95bfe5c58d9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.167126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5a9f8-9dc3-4877-bd4b-356262727ce6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.168769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 585/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:47:07 2022] GET /identity/v3/projects/a175feac96244075a51ee95bfe5c58d9 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:07.174845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.201969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:07.282877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2235a4-d990-4d72-96a7-2fe92c926692 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.317962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8ac22-33b0-4d6a-85b3-7885648e08c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.318608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8ac22-33b0-4d6a-85b3-7885648e08c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.318890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8ac22-33b0-4d6a-85b3-7885648e08c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.319164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8ac22-33b0-4d6a-85b3-7885648e08c8 None admin] PATH_INFO: `/v3/projects/a175feac96244075a51ee95bfe5c58d9/users/834dddb604f14873b49b86724b8a2e1b/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.321405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8ac22-33b0-4d6a-85b3-7885648e08c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a175feac96244075a51ee95bfe5c58d9, user_id=834dddb604f14873b49b86724b8a2e1b, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.322845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8ac22-33b0-4d6a-85b3-7885648e08c8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.335272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 607/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:47:07 2022] PUT /identity/v3/projects/a175feac96244075a51ee95bfe5c58d9/users/834dddb604f14873b49b86724b8a2e1b/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:47:09.471950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7fb8-0e69-40e8-8aeb-fa63ad433e3b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:09.472470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7fb8-0e69-40e8-8aeb-fa63ad433e3b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:09.472470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e7fb8-0e69-40e8-8aeb-fa63ad433e3b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:09.473803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 586/1193] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:09.479219 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:09.479651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:09.479789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:09.497647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:09.498819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:09.563621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 608/1194] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:09.832799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7fb8-0e69-40e8-8aeb-fa63ad433e3b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:09.933008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e7fb8-0e69-40e8-8aeb-fa63ad433e3b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:09.966561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:09.967183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:09.967493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:09.967764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:09.969557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:09.978325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:47:09.983166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:47:09.984268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:47:09.985419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 587/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:47:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:47:09.990290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.004758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.090541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e5b01-9dad-4e80-8ab6-1bc186b83d83 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.124287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.124889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.125168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.125450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.126204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:10.126587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.128292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.136773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 609/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:10.142786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.241015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f2957-34ff-4902-ba8d-8ffef032a537 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.276736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a639a-ab34-417a-9e69-956373610265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.277460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a639a-ab34-417a-9e69-956373610265 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.277630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a639a-ab34-417a-9e69-956373610265 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.277906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a639a-ab34-417a-9e69-956373610265 None admin] PATH_INFO: `/v3/users/834dddb604f14873b49b86724b8a2e1b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.279125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a639a-ab34-417a-9e69-956373610265 None admin] RBAC: Authorizing `identity:get_user(user_id=834dddb604f14873b49b86724b8a2e1b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.280586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a639a-ab34-417a-9e69-956373610265 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.282169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 588/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/users/834dddb604f14873b49b86724b8a2e1b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:10.288324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.365824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.390737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea43c2-d632-4a03-8494-b3a1c36fb56d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.422946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.423577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.423849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.424119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.491140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.492629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.564289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.589379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 610/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:47:10.595379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a639a-ab34-417a-9e69-956373610265 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.655827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.693695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a639a-ab34-417a-9e69-956373610265 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.726839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f05d29-8b66-4eee-9f0b-d871238103cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.728099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f05d29-8b66-4eee-9f0b-d871238103cc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.728374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f05d29-8b66-4eee-9f0b-d871238103cc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.728639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f05d29-8b66-4eee-9f0b-d871238103cc None admin] PATH_INFO: `/v3/projects/a175feac96244075a51ee95bfe5c58d9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.729745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f05d29-8b66-4eee-9f0b-d871238103cc None admin] RBAC: Authorizing `identity:get_project(project_id=a175feac96244075a51ee95bfe5c58d9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.731168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f05d29-8b66-4eee-9f0b-d871238103cc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.732831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 589/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/projects/a175feac96244075a51ee95bfe5c58d9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:10.738954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.835024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c3a28-18ab-47f7-aa14-0458bd49d0bb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.869059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.869654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.869931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.870202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.870910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=a175feac96244075a51ee95bfe5c58d9, user.id=834dddb604f14873b49b86724b8a2e1b) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:10.871235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.872661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70c57b-0784-4c1e-89b3-2b882d2b099a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.878355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 611/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/role_assignments?user.id=834dddb604f14873b49b86724b8a2e1b&scope.project.id=a175feac96244075a51ee95bfe5c58d9&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:20.722713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=35 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:20.875710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=81 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:07.665779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e5dc8-aec6-4b09-88ca-b71384660d6e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:07.665779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e5dc8-aec6-4b09-88ca-b71384660d6e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:07.665779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e5dc8-aec6-4b09-88ca-b71384660d6e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:07.666851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 590/1201] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:07.673314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:07.674505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:07.674983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:07.698071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:07.699492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:07.830964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 612/1202] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:08.150859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e5dc8-aec6-4b09-88ca-b71384660d6e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:08.307129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e5dc8-aec6-4b09-88ca-b71384660d6e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:08.353760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:08.354515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:08.354800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:08.355063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:08.368788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:08.370275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:08.388372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:08.390593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:08.392897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:08.392897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:08.392897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:08.396645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:08.399174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 591/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:08.606004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf076b-952e-4d81-b435-4d426d8b2eec None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:08.644006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:08.652252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:08.652907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:08.653197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:08.653446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:08.654120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:08.654468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:08.656003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe04fa7-35f5-481b-9c77-ce9b1944ebce None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:08.671073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 613/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:08.678353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:08.773979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:08.848692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2564e-8c3f-40c9-a0d9-4b91a41c17e8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:08.893427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ec93-1990-4ce0-94dd-0cf2cf99004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:08.899245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ec93-1990-4ce0-94dd-0cf2cf99004b None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:08.899245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ec93-1990-4ce0-94dd-0cf2cf99004b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:08.899245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ec93-1990-4ce0-94dd-0cf2cf99004b None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:08.899245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8ec93-1990-4ce0-94dd-0cf2cf99004b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:08.899245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8ec93-1990-4ce0-94dd-0cf2cf99004b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:08.931040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 592/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Sep 22 14:48:08 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:48:11.651093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb60a88-2b98-46ac-ab10-bec4865ec67a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:11.651715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb60a88-2b98-46ac-ab10-bec4865ec67a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:11.651795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb60a88-2b98-46ac-ab10-bec4865ec67a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:11.653269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 614/1206] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:11.660787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:11.661457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:11.661457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:11.683308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:11.684690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:11.771744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 593/1207] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:12.068200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb60a88-2b98-46ac-ab10-bec4865ec67a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.213799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb60a88-2b98-46ac-ab10-bec4865ec67a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.257579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.258293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.258570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.258835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.271273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:12.276559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:12.277793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:12.278997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 615/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:12.284079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.329491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.416679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f12cd-1b56-447c-9a63-36686d7efbfd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.459354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.460702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.461017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.461352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.462179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:12.462522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.464102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.473370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 594/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:12.482623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.556366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.621958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848678b9-f231-4803-a75c-1d07c9eed18e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.668178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.668986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.669197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.669465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.681879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.685634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:12.701379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:12.702750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:12.703691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:12.703691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:12.703691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:12.703691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 616/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:12.709571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.857008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.861638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f882a-f5a3-4061-ae9b-f38a1be790ac None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.928296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.928980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.929349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.929734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.930496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:12.930926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.932925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.952286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 595/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:12.958503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.113233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3078f1-e659-4c45-abd7-22df1262c399 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.115934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:13.173845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.175424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.176174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.176634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.191056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.192549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:13.212366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:13.213441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:13.214436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:13.216981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 617/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:13.224536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.400350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ae962-c0fb-40fd-b713-c34df14959f4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.459923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.460480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.460854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.461060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.461957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:13.462276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.463770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.486252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 596/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:13.492856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.622985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5d9a6-5186-438b-9d14-7ad32556731f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.674594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.675490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.675595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.675815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.716696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:13.790263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.792032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.949812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 618/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:48:13.957039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.991592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:14.106702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e3a7-5540-41ef-92b1-e9c503bde453 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:14.167521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:14.168298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:14.168608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:14.168883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:14.190743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:14.192246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:14.212508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:14.214160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:14.215147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:14.215147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:14.215147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:14.219447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 597/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:14.220914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:14.299720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:14.358177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727365c1-75cf-4170-8146-409819c26a3d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:14.418648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:14.419297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:14.419559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:14.419946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:14.420471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:14.420770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:14.422171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:14.440497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 619/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:14.442064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:14.574375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:14.617070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd3481-e1e6-4d01-9a2b-579fa191c9bf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:14.655859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:14.656521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:14.656901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:14.657239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:14.664910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:14.666545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:14.679649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:14.681352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:14.682321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 598/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:14.687913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:14.837601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946b0c8-5b73-467e-a9be-687c7f8e39e7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:14.874660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:14.893088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:14.893991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:14.894510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:14.894958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:14.896523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:14.896903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:14.898803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1ae51-c457-4f2d-975c-fb47d2fce7eb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:14.914970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 620/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:14.922344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:15.068882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edbb27-8b34-47f4-8872-3bf3697b6555 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:15.115101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:15.124137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:15.125268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:15.125802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:15.126317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:15.127152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=a3f43e7a1e9840299697271428a67872) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:15.127642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:15.130095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d20bf-9dda-4257-94fb-2e17e44dbdd9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:15.155233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 599/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:14 2022] GET /identity/v3/role_assignments?user.id=a3f43e7a1e9840299697271428a67872&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:17.618663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca49ff-6610-43c3-8318-8d885200080e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.619363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca49ff-6610-43c3-8318-8d885200080e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.619363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca49ff-6610-43c3-8318-8d885200080e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.621292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 621/1220] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:17.627318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.627844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.627998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.653281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:17.654984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:17.745989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 600/1221] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:18.062584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca49ff-6610-43c3-8318-8d885200080e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.196017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca49ff-6610-43c3-8318-8d885200080e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.249175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.250142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.250509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.250855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.266531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:18.274714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:18.276536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:18.278164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 622/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:18.284763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.427289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a4973-1171-4548-9593-aa8fa6dffaf8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.458030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:18.475799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.476480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.476760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.477114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.477984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:18.478424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.479951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.492670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 601/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:18.501798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.572287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:18.650253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade382e-5a6f-42e8-ad6b-257cef82fb5f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.695323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.695862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.696135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.696402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.711379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.712933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:18.730479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:18.738980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:18.740518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:18.740518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:18.740518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:18.740518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 623/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:18.741743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.868382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584bd9-d5e7-41a1-9f80-54920a3dbcda None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.908953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.909872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.910367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.910939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.912163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:18.913223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.915111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.936556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 602/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:18.937901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:19.073626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:19.086618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac8593-660d-4de5-9fa6-ecc6db50a565 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:19.131779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:19.132747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:19.133052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:19.133371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:19.144285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:19.145969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:19.160618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:19.161871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:19.163762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 624/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:19.169731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:19.232692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:19.298075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10115f64-37d8-49d4-af7f-849cbc1ca920 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:19.339084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:19.339665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:19.340844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:19.340844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:19.341054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:19.341675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:19.343440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:19.366416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 603/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:19.373476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:19.501686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57072a-d795-41c1-8210-e2f784bb1ed6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:19.540020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:19.540020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:19.540587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:19.540587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:19.552585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:19.554711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:19.570332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:19.571429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:19.572660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:19.572660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:19.572660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:19.572660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 625/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:19.577726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:19.712482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b91338-0519-450f-8b7d-94263ba29134 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:19.715845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:19.755289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:19.756145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:19.756354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:19.756633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:19.757545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:19.757848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:19.759391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:19.773457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 604/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:19.779592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:19.790535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:19.914988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116dd9-53bf-4c59-93f4-370909b970e3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:19.957529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:19.958204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:19.958472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:19.958781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:19.969102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:19.970554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:19.985004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:19.986552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:19.987855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 626/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:19.993531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:20.153991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e544-550f-46aa-9f96-a5d690182111 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:20.198660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:20.199313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:20.199596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:20.199896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:20.200604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:20.200916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:20.202371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e51c0-c2c0-4803-8e50-ab479716c0a5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:20.216785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 605/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:20.222956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:20.318010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:20.362872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1ec80-1497-4a6e-8423-c4046a26d1ff None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:20.415100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117de2d5-d36e-4a86-99e2-3552f746d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:20.415843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117de2d5-d36e-4a86-99e2-3552f746d31a None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:20.416123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117de2d5-d36e-4a86-99e2-3552f746d31a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:20.416396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117de2d5-d36e-4a86-99e2-3552f746d31a None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/a3f43e7a1e9840299697271428a67872/roles/56e07737e089485e80f6e9a159e146c6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:20.436912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117de2d5-d36e-4a86-99e2-3552f746d31a None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=a3f43e7a1e9840299697271428a67872, role_id=56e07737e089485e80f6e9a159e146c6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:20.439617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117de2d5-d36e-4a86-99e2-3552f746d31a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:20.458844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 627/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:48:20 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/a3f43e7a1e9840299697271428a67872/roles/56e07737e089485e80f6e9a159e146c6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:48:22.879107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa679-2199-42eb-b8fc-745fac01f220 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:22.879730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa679-2199-42eb-b8fc-745fac01f220 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:22.879730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755aa679-2199-42eb-b8fc-745fac01f220 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:22.881083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 606/1233] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:22.888529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:22.889076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:22.889406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:22.913415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:22.915424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:23.008994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 628/1234] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:23.347562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aa679-2199-42eb-b8fc-745fac01f220 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.377434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:23.491607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755aa679-2199-42eb-b8fc-745fac01f220 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.552489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.553604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.554101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.554582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.569661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:23.575542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:23.577449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:23.579236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 607/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:23.585083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.641807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:23.721659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22278a76-fecf-488e-a00f-e341bd55941c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.775071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.775889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.776376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.777130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.778045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:23.778387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:23.781271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:23.794107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 629/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:23.800512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.929613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:23.948300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ea663-51f8-447b-bbbd-02d3ecbc4533 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.995509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.996375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.996780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.997175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:24.016266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:24.018842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:24.040752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:24.042715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:24.045107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:24.045107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:24.045107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:24.045107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 608/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:24.051975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:24.213346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588135d-2e8b-4aa8-b096-b81b0d7191a7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:24.224086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:24.262466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:24.263154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:24.263559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:24.263872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:24.264669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:24.265042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:24.267383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:24.289835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 630/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:24.296649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:24.460400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bec97-23fc-43e9-bb57-6e8115fc6b0c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:24.521293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:24.522166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:24.522536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:24.522945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:24.526779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:24.533829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:24.536615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:24.559167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:24.560337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:24.561413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:24.567749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 609/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:24.570091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:24.726183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435d19-2a35-4aa7-95a1-3c95cf98a118 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:24.776654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:24.777789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:24.778223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:24.778677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:24.780046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:24.780580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:24.782747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:24.821238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 631/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:24.822723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:24.988668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285578eb-ae4a-426f-aef7-c7dcbc5a40b4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:25.040058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:25.040810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:25.041136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:25.041457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:25.153945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:25.170629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:25.172264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:25.351765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 610/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:48:25.359119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:25.419762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:25.556409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a99a20-1e37-4375-802a-4807bc71b861 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:25.608264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:25.608835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:25.609123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:25.609383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:25.624480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:25.626683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:25.645758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:25.648144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:25.650126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:25.650126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:25.650126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:25.650126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 632/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:25.655582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:25.794493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:25.805701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8c31a-ded5-4b10-b651-f59c7edd0e96 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:25.856071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:25.856756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:25.857074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:25.857425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:25.858139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:25.858565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:25.860052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:25.874863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 611/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:25.881101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:26.004683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:26.042438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f22159-5e3e-46db-b6c2-3e8776d699e2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:26.096112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047817-8804-47dc-8606-f84a1314437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:26.096785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047817-8804-47dc-8606-f84a1314437b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:26.097068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047817-8804-47dc-8606-f84a1314437b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:26.097343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db047817-8804-47dc-8606-f84a1314437b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:26.106906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db047817-8804-47dc-8606-f84a1314437b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:26.108461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db047817-8804-47dc-8606-f84a1314437b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-db047817-8804-47dc-8606-f84a1314437b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:26.123154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:26.124897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:26.126599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 633/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:26.131764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:26.311633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762088c-fd90-4e8a-aecc-94f621bd3c02 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:26.364467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:26.365299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:26.365877 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:26.366257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:26.367394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:26.367924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:26.369879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35116654-c22d-4b76-9fb7-1ee4fa21652c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:26.390062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 612/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:26.399378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047817-8804-47dc-8606-f84a1314437b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:26.550509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db047817-8804-47dc-8606-f84a1314437b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:26.605183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:26.606011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:26.606368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:26.606702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:26.607715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] RBAC: Adding query filter params (role.id=56e07737e089485e80f6e9a159e146c6, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=a3f43e7a1e9840299697271428a67872) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:26.608186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:26.609950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eec62-eec6-4092-a7dc-ab55c231a78c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:26.619703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 634/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:26 2022] GET /identity/v3/role_assignments?user.id=a3f43e7a1e9840299697271428a67872&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=56e07737e089485e80f6e9a159e146c6 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:29.368186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409c41-7cb4-43f9-b167-51a3ae5929e1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:29.369418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409c41-7cb4-43f9-b167-51a3ae5929e1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:29.370068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f409c41-7cb4-43f9-b167-51a3ae5929e1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:29.372999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 613/1247] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:29.379571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:29.380407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:29.380407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:29.387836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:29.407609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:29.407609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:29.519116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 635/1248] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:29.812553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409c41-7cb4-43f9-b167-51a3ae5929e1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:29.838955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:29.958037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f409c41-7cb4-43f9-b167-51a3ae5929e1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:30.001900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:30.002547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:30.002825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:30.003086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:30.016763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:30.023186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:30.024241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:30.025104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 614/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:30.031705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:30.172043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ff4f-0af7-43fe-8c21-a7e49abadfc9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:30.222352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:30.229956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:30.230707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:30.233165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:30.233165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:30.233165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:30.233495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:30.234934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:30.248784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 636/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:48:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:30.258089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:30.374717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:30.400638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5850ad2-4417-4ad8-b634-9436ab2cdc40 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:30.445635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:30.446193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:30.446545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:30.446873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:30.461073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:30.462663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:30.480915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:30.482658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:30.483875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:30.483875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:30.483875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:30.483875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 615/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:30.488237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:30.651234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5b630-4d5c-4bf5-a981-a7ba56c681cb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:30.697951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:30.698612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:30.700339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:30.700339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:30.700339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:30.700339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:30.702652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:30.718205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 637/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:30.724198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:30.855870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0b10d-cc9a-4b42-9782-32d13814701e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:30.898713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:30.918416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:30.921087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:30.921087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:30.921087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:30.937059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:30.938662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:30.956332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:30.959856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:30.960914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 616/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:30.965945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.047344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:31.099731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07fb6-57c4-441a-b4e1-0bae59d529cb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:31.140807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:31.141418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.141752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.141982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.142737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:31.143044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:31.144553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:31.164819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 638/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:31.171189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.295358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66937767-96bd-451e-8be1-88ae2dc15a5b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:31.348068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:31.348729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.349024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.349314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.424084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:31.442921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:31.445771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:31.586560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 617/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:48:31.595493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.748589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216093c-296a-48e2-ad9f-90922b659f14 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:31.771561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:31.806355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:31.807452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.807936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.808412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.824656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:31.826141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:31.842259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:31.843566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:31.844486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:31.844486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:31.844486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:31.844486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 639/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:31.852050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.994424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.009381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244da70-30a6-4b10-bf2b-a095d26539c9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.057643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.058298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.058571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.058880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.059547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:32.059847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.061263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.076374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 618/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:32.082669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.248404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dfdd9-95e4-4918-86c4-e1ec9ece9038 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.293480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.294151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.294431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.294705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.302773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.304372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:32.317635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:32.320038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:32.322156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 640/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:32.327513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.484734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330cfab-25b9-4178-a499-3f051c3b5e59 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.523818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.524367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.524653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.524930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.525642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:32.525947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.527443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42af35-1974-4850-9939-3d5726cc97b8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.542066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 619/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:32.548373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.590764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.712422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ded3-cddd-40f8-b5c2-c4224fc52729 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.778586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.779602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.780119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.780560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.781965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=a3f43e7a1e9840299697271428a67872) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:32.782570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.785687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f26f8d-e2d0-4f67-a3c8-57ddbea05721 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.797375 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 641/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/role_assignments?user.id=a3f43e7a1e9840299697271428a67872&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:35.011027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061765a-b6ea-47e1-ba82-0f8f57b0581f None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.011618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061765a-b6ea-47e1-ba82-0f8f57b0581f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.011618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061765a-b6ea-47e1-ba82-0f8f57b0581f None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.012936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 620/1261] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:35.018049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.018470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.018608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.038481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:35.039719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:35.107077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 642/1262] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:35.399680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061765a-b6ea-47e1-ba82-0f8f57b0581f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:35.410654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:35.505403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061765a-b6ea-47e1-ba82-0f8f57b0581f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:35.546981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:35.547629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.547907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.548176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.557999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:35.562809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:35.563952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:35.564815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 621/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:48:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:35.570657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:35.589097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:35.669797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9faa7-6357-43dd-9d4f-41f8d0b6634a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:35.709638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:35.710238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.710510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.710784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.711468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:35.711763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:35.715687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:35.726344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 643/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:48:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:35.732230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:35.799105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:35.833349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193b15-5209-458f-aa66-069a63414a87 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:35.867224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:35.867864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.868140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.868413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.878056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:35.879526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:35.894813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:35.896027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:35.897026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:35.897026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:35.897026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:35.897026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 622/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:35.901972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:35.981034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.005634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9af48a-3943-4729-8f14-8faf085ce3d2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.039730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.040409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.040591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.040862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.041495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:36.041801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.043169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.055355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 644/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:36.061212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.163337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd9b8-f62a-4f67-b5ba-0bdaee5a6db8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.188182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.201958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.202584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.202855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.203124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.210421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.211881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:36.222640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:36.223824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:36.225463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 623/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:36.230020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.337861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d7534-245c-4df5-a0a4-16c231e040d6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.372651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.373248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.373530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.373804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.374523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:36.374833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.376325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.385033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.393742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 645/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:36.400312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.497540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf08c0-4aa9-4791-bf87-a375783a79a6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.538534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.539155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.539467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.539739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.550216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.551688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:36.566802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:36.567962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:36.569045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:36.569045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:36.569045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:36.569045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 624/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:36.573881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.676317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8b982-e096-4d5b-b4c1-1d741c940f6e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.710241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.710848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.711134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.711435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.712069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:36.712374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.713751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.725670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 646/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:36.732486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.764222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.830795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9463ea1-0595-4bc1-a380-6ecb4b31720e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.863405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.864029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.864313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.864582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.871450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.872892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:36.885078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:36.886790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:36.888540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 625/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:36.892177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.953365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.995778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb9523-8cb2-4a37-a54d-8de5e216da87 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.030734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.031366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.031644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.031944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.032620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:37.032920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.034340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1091d-4c51-45e3-b99f-72561d84ae5c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.046157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 647/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:37.052338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:37.145559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:37.157368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c5742-b1ff-4ba4-8602-ef9c0379085e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.191249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cca629-eb87-4988-81eb-22c5bc19c64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.191892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cca629-eb87-4988-81eb-22c5bc19c64a None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.192165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cca629-eb87-4988-81eb-22c5bc19c64a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.192432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cca629-eb87-4988-81eb-22c5bc19c64a None admin] PATH_INFO: `/v3/projects/83c0de024d3b4120a69df2088bded628/users/a3f43e7a1e9840299697271428a67872/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.194862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cca629-eb87-4988-81eb-22c5bc19c64a None admin] RBAC: Authorizing `identity:create_grant(project_id=83c0de024d3b4120a69df2088bded628, user_id=a3f43e7a1e9840299697271428a67872, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.196309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cca629-eb87-4988-81eb-22c5bc19c64a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.208431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 626/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Sep 22 14:48:37 2022] PUT /identity/v3/projects/83c0de024d3b4120a69df2088bded628/users/a3f43e7a1e9840299697271428a67872/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:48:39.360302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304711-8216-4d68-9d2d-27b7acedadb0 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:39.360895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304711-8216-4d68-9d2d-27b7acedadb0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:39.360895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d304711-8216-4d68-9d2d-27b7acedadb0 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:39.362234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 648/1274] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:39.369060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:39.369630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:39.369630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:39.389947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:39.391171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:39.461216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 627/1275] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:39.736182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304711-8216-4d68-9d2d-27b7acedadb0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:39.840803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d304711-8216-4d68-9d2d-27b7acedadb0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:39.858665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:39.876243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:39.876865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:39.877143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:39.877417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:39.888529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:39.893379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:39.894457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:39.900805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 649/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Sep 22 14:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:39.901694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:39.978924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:39.999473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa755c-d15e-47ee-9893-f76a76971075 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.033823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.034419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.034690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.034955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.035636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:40.035936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.037786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.045559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 628/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Sep 22 14:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:40.051658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.152164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909d89-0d89-4ad8-88b3-6ec0e1de6799 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.193468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.194266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.194405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.194643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.206884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.209074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:40.228409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:40.229634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:40.230761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:40.230761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:40.230761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:40.230761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 650/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:40.235503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.332014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6b196-8f05-4839-95d0-76cc492c565b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.370122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.370832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.370986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.371282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.371914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:40.372217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.373572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.385159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 629/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:40.391061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.422751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:40.499516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36aec4-5844-4ed9-a737-7a16fc032854 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.535006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.535664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.535945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.536220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.544044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.545492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:40.556805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:40.558252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:40.559503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 651/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:40.563953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.568443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:40.664491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753bf3e-dc51-4f87-9e7f-1a12f3e33d23 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.697807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.698510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.700289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.700289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.700289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:40.700289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.702074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.722528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 630/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:40.728454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.826646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:40.836558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d20ace-3409-47ba-b1d5-55d13303bb7b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.868890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.869485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.869760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.870025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.938710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.940102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:41.025621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:41.036725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 652/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:48:41.042558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:41.102272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:41.142734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c0888-3bf8-4f82-b76d-996c9dd8e9b5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:41.176749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:41.177883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:41.178161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:41.178433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:41.188089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:41.189567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:41.204092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:41.205224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:41.206492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:41.206492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:41.206492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:41.206492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 631/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:41.210985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:41.309283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba029-6002-45f7-be07-de7e0773f623 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:41.341753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:41.342350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:41.342629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:41.342900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:41.343560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:41.343865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:41.345233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:41.356751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 653/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:41.362444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:41.461333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470437-b947-4bf8-be7d-377e5db82380 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:41.474747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:41.494761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:41.495413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:41.495686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:41.495952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:41.502290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:41.503778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:41.515555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:41.516683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 22 14:48:41.517809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 632/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:41.522730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:41.574418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:41.625666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1033d-1cef-4d23-b45b-0b4a39692a59 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:41.659763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:41.660371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:41.660653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:41.660927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:41.661684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:41.662495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:41.663961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaa524-c521-479d-853c-c99dc49eb0e0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:41.676339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 654/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:41.682683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:41.787584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f938aa-f0f7-4a55-b806-68108eb10a7e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:41.820358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:41.820969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:41.821246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:41.821517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:41.822233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] RBAC: Adding query filter params (role.id=5519f8e533ba4c5b8d81cdb709b861be, scope.project.id=83c0de024d3b4120a69df2088bded628, user.id=a3f43e7a1e9840299697271428a67872) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:41.822539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:41.823998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186e27c-1688-4dcc-9c48-3fbdf04ebd7a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:41.830056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 633/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:41 2022] GET /identity/v3/role_assignments?user.id=a3f43e7a1e9840299697271428a67872&scope.project.id=83c0de024d3b4120a69df2088bded628&role.id=5519f8e533ba4c5b8d81cdb709b861be => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:44.062363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ce8fe-a94b-4e0d-8539-eead44be0e4c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:44.062933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ce8fe-a94b-4e0d-8539-eead44be0e4c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:44.062933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ce8fe-a94b-4e0d-8539-eead44be0e4c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:44.064368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 655/1288] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:44.071148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:44.071804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:44.071804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:44.094398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:44.098396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:44.131544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:44.174508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 634/1289] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:44.453049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ce8fe-a94b-4e0d-8539-eead44be0e4c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:44.562262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ce8fe-a94b-4e0d-8539-eead44be0e4c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:44.573506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:44.598161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:44.598798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:44.599628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:44.599906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:44.600495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:44.601878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:44.610830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:44.612001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:44.612863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 656/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:48:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:44.618723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:44.720096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cf758-405c-4b9d-a29e-24e58520c298 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:44.754259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:44.754858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:44.755156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:44.755466 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:44.756086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:44.756387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:44.758003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67933a-cc8d-4555-8c45-4cc5f954acb2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:44.762697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 635/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:44.768479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:44.866636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e615a-82c0-4c39-9d08-73c211fc166a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:44.904863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:44.905526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:44.905762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:44.906037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:44.906666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:44.906971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:44.908376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9326ef09-2dfa-4e62-8fd0-2c4956230028 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:44.913299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 657/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:47.061534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e51c-adf7-4260-8a10-0253d96d8c67 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:47.062130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e51c-adf7-4260-8a10-0253d96d8c67 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:47.062130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e51c-adf7-4260-8a10-0253d96d8c67 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:47.063446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 636/1293] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:47.069436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268178a6-a861-47cc-a7e2-27029ad9c18b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:47.069923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268178a6-a861-47cc-a7e2-27029ad9c18b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:47.070026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268178a6-a861-47cc-a7e2-27029ad9c18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:47.087905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-268178a6-a861-47cc-a7e2-27029ad9c18b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:47.089093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-268178a6-a861-47cc-a7e2-27029ad9c18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:47.118725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:47.161737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 658/1294] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:47.439295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973e51c-adf7-4260-8a10-0253d96d8c67 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:47.472891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:47.547182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973e51c-adf7-4260-8a10-0253d96d8c67 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:47.580247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5c81-ab7b-48fa-8c22-92d5956ff761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:47.580849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5c81-ab7b-48fa-8c22-92d5956ff761 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:47.581129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5c81-ab7b-48fa-8c22-92d5956ff761 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:47.581405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5c81-ab7b-48fa-8c22-92d5956ff761 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:47.582053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5c81-ab7b-48fa-8c22-92d5956ff761 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:47.583462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5c81-ab7b-48fa-8c22-92d5956ff761 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:47.598559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 637/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Sep 22 14:48:47 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:48:49.731958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c306b20-a45a-40e4-92b8-4b658c64d9c9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:49.732486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c306b20-a45a-40e4-92b8-4b658c64d9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:49.732486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c306b20-a45a-40e4-92b8-4b658c64d9c9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:49.733756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 659/1296] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:49.739834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:49.740384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:49.741084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:49.759735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:49.760938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:49.831610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 638/1297] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:50.113734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c306b20-a45a-40e4-92b8-4b658c64d9c9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:50.230132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c306b20-a45a-40e4-92b8-4b658c64d9c9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:50.262924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:50.263564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:50.263839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:50.264109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:50.264683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:50.266036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:50.277926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:50.279477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:50.280517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 660/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:48:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:50.285055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:50.303984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:50.382659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f216164-9449-440c-889d-6c0d368d5839 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:50.415031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:50.415654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:50.415933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:50.416203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:50.416821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:50.417120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:50.418486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:50.423611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 639/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:50.430405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:50.461803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:50.531898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c45de-5741-4755-8324-b463900fb2d5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:50.565377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:50.565989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:50.566269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:50.566540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:50.567258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] RBAC: Adding query filter params (interface=public, service_id=0f9fa6112f9841e6806b43e17884fb1b, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:50.567571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:50.568937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad61afb-2892-46f0-a42c-1b4121f9e9a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:50.573920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 661/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Sep 22 14:48:50 2022] GET /identity/v3/endpoints?service_id=0f9fa6112f9841e6806b43e17884fb1b&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:50.579673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:50.659057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:50.677290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56b2c-def3-4dd7-8e15-4602a954c37c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:50.710243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cbc55-bb84-4e02-a95c-0a449dc6ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:50.710842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cbc55-bb84-4e02-a95c-0a449dc6ce53 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:50.711124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cbc55-bb84-4e02-a95c-0a449dc6ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:50.711428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cbc55-bb84-4e02-a95c-0a449dc6ce53 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:50.712026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cbc55-bb84-4e02-a95c-0a449dc6ce53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:50.713393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cbc55-bb84-4e02-a95c-0a449dc6ce53 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:50.719968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 640/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Sep 22 14:48:50 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:48:52.826474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3692-426c-4cb6-a267-bafafe35b9fb None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:52.827019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3692-426c-4cb6-a267-bafafe35b9fb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:52.827019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f3692-426c-4cb6-a267-bafafe35b9fb None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:52.828386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 662/1302] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:52.834097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:52.834515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:52.834707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:52.852637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:52.853830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:52.919413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 641/1303] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:53.185986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f3692-426c-4cb6-a267-bafafe35b9fb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:53.282229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:53.289978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f3692-426c-4cb6-a267-bafafe35b9fb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:53.322584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:53.323231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:53.323498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:53.323762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:53.324333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:53.325678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:53.334436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:53.335695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 22 14:48:53.336602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 663/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Sep 22 14:48:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:53.341663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:53.415788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:53.439370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10f959-2545-4c2b-bebb-9667bd063a11 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:53.472146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:53.472751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:53.473030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:53.473305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:53.474585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] RBAC: Adding query filter params (name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:53.474765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:53.476169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd1f5b-4202-4bf9-b1e2-ae9ca11104ed None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:53.484004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 642/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:53.490281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:53.588732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72096d0f-50c7-459d-8092-170c8b054a3c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:53.622761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f260ce-5f1f-459a-a082-0c53ae89b00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:53.623387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f260ce-5f1f-459a-a082-0c53ae89b00e None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:53.623665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f260ce-5f1f-459a-a082-0c53ae89b00e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:53.623931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f260ce-5f1f-459a-a082-0c53ae89b00e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:53.624591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f260ce-5f1f-459a-a082-0c53ae89b00e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:53.625942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f260ce-5f1f-459a-a082-0c53ae89b00e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:53.644746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 664/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:48:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:49:03.478446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=30 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:03.640884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:58.108324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb36209-19b3-4675-9497-1f3a8c1d61c4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:58.108777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb36209-19b3-4675-9497-1f3a8c1d61c4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:58.108989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb36209-19b3-4675-9497-1f3a8c1d61c4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:58.110212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 643/1307] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:49:58.116060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be455-574b-4d03-a81e-fdbb5b257db6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:58.116447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be455-574b-4d03-a81e-fdbb5b257db6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:58.116601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000be455-574b-4d03-a81e-fdbb5b257db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:58.220209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-000be455-574b-4d03-a81e-fdbb5b257db6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:49:58.221431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-000be455-574b-4d03-a81e-fdbb5b257db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:58.332863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 665/1308] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:49:58.630701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3491-f8f0-4025-a8e4-14f2d38c2bc8 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:58.631323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3491-f8f0-4025-a8e4-14f2d38c2bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:58.631971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb3491-f8f0-4025-a8e4-14f2d38c2bc8 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:58.633444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 644/1309] 10.222.0.66 () {54 vars in 964 bytes} [Thu Sep 22 14:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:49:58.639496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1751-28eb-4f41-9500-2a4b47af2dff None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:58.640003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1751-28eb-4f41-9500-2a4b47af2dff None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:58.640377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1751-28eb-4f41-9500-2a4b47af2dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:58.728744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-00ae1751-28eb-4f41-9500-2a4b47af2dff None None] MFA Rules not processed for user `60c809b5a47a4983be26116553934387`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:49:58.730049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-00ae1751-28eb-4f41-9500-2a4b47af2dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:58.755934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:58.828172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 666/1310] 10.222.0.66 () {58 vars in 1061 bytes} [Thu Sep 22 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:49:58.838310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb3491-f8f0-4025-a8e4-14f2d38c2bc8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:49:58.840100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4acb3491-f8f0-4025-a8e4-14f2d38c2bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:58.983487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb3491-f8f0-4025-a8e4-14f2d38c2bc8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:49:59.071752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:49:59.072490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:59.072775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:59.073047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:59.074645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:59.138017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:59.204630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:49:59.206541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d6bc9a-a973-4ffd-a7b5-f2d7c881b0eb service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:49:59.228976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 645/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:01.726826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495ed5-8840-4e04-b284-5bac51b0749f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:01.727285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495ed5-8840-4e04-b284-5bac51b0749f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:01.727423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01495ed5-8840-4e04-b284-5bac51b0749f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:01.728790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 667/1312] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:01.734122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717fbb-376e-41e3-a759-102578f2cada None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:01.734640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717fbb-376e-41e3-a759-102578f2cada None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:01.735030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7717fbb-376e-41e3-a759-102578f2cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:01.822618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a7717fbb-376e-41e3-a759-102578f2cada None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:01.823865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a7717fbb-376e-41e3-a759-102578f2cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:01.919050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 646/1313] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:02.205115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01495ed5-8840-4e04-b284-5bac51b0749f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:02.206655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-01495ed5-8840-4e04-b284-5bac51b0749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:02.309605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01495ed5-8840-4e04-b284-5bac51b0749f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:02.381703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:02.384641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:02.385441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:02.385939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:02.386269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:02.388090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:02.495903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:02.497345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc927b-2f3a-4aad-ba84-6471ad698b94 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:02.515450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 668/1314] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:04.992432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1796c-c08f-4148-bc95-60ed6b034031 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:04.992960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1796c-c08f-4148-bc95-60ed6b034031 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:04.993078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1796c-c08f-4148-bc95-60ed6b034031 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:04.994336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 647/1315] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:04.999671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521e1f-e04c-4996-a6de-fca808e33e40 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:05.000066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521e1f-e04c-4996-a6de-fca808e33e40 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:05.000229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521e1f-e04c-4996-a6de-fca808e33e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:05.066934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7d521e1f-e04c-4996-a6de-fca808e33e40 None None] MFA Rules not processed for user `a775d9fba1794acc91783971d2655e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:05.068253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7d521e1f-e04c-4996-a6de-fca808e33e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:05.119312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:05.165935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 669/1316] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:05.455849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1796c-c08f-4148-bc95-60ed6b034031 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:05.457373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fda1796c-c08f-4148-bc95-60ed6b034031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:05.467105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:05.566332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1796c-c08f-4148-bc95-60ed6b034031 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:05.638850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:05.639478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:05.639759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:05.640029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:05.641627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:05.703176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:05.750818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:05.752299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5641c3-06ad-4391-adc3-9b7990884dd4 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:05.770577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 648/1317] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:08.483711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5537f95-f39a-4843-b28b-d16e23c264ae None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:08.484246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5537f95-f39a-4843-b28b-d16e23c264ae None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:08.484346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5537f95-f39a-4843-b28b-d16e23c264ae None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:08.485678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 670/1318] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:08.492002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da5b6-258a-4a73-af5f-575af49508d1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:08.492432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da5b6-258a-4a73-af5f-575af49508d1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:08.492542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76da5b6-258a-4a73-af5f-575af49508d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:08.562597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d76da5b6-258a-4a73-af5f-575af49508d1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:08.563817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d76da5b6-258a-4a73-af5f-575af49508d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:08.661108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 649/1319] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:08.964013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5537f95-f39a-4843-b28b-d16e23c264ae None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:08.965719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e5537f95-f39a-4843-b28b-d16e23c264ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:09.075169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5537f95-f39a-4843-b28b-d16e23c264ae None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:09.156253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:09.156866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:09.157145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:09.157417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:09.159055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:09.166070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:09.269972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:09.271365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34664a-d4df-4261-bf69-f35a68e2800e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:09.289930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 671/1320] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:12.144250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3f889-abd8-4127-8c23-dbd952d9fbeb None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:12.144812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3f889-abd8-4127-8c23-dbd952d9fbeb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:12.145078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3f889-abd8-4127-8c23-dbd952d9fbeb None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:12.146926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 650/1321] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:12.153613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc21a4-d2be-4bd6-849e-0f2fc9d1f5e6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:12.154244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc21a4-d2be-4bd6-849e-0f2fc9d1f5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:12.154330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc21a4-d2be-4bd6-849e-0f2fc9d1f5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:12.228681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f5fc21a4-d2be-4bd6-849e-0f2fc9d1f5e6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:12.229922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f5fc21a4-d2be-4bd6-849e-0f2fc9d1f5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:12.292793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:12.334551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 672/1322] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:50:12.634704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e33b0-52ec-4c1a-91c9-d1390270c608 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:12.634987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e33b0-52ec-4c1a-91c9-d1390270c608 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:12.635288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e33b0-52ec-4c1a-91c9-d1390270c608 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:12.636643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 651/1323] 10.222.0.66 () {54 vars in 964 bytes} [Thu Sep 22 14:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:50:12.642970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ae1af-a49f-4126-86c6-3946828cf359 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:12.643396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ae1af-a49f-4126-86c6-3946828cf359 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:12.643780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ae1af-a49f-4126-86c6-3946828cf359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:12.720051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-761ae1af-a49f-4126-86c6-3946828cf359 None None] MFA Rules not processed for user `60c809b5a47a4983be26116553934387`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:12.721256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-761ae1af-a49f-4126-86c6-3946828cf359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:12.803772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 673/1324] 10.222.0.66 () {58 vars in 1061 bytes} [Thu Sep 22 14:50:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:50:12.812654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e33b0-52ec-4c1a-91c9-d1390270c608 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:12.814198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c78e33b0-52ec-4c1a-91c9-d1390270c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:12.834343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:12.926480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e33b0-52ec-4c1a-91c9-d1390270c608 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:13.004844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:13.005491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:13.005749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:13.005979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:13.007840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:13.082495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:13.121203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:13.122646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012f88b-10a9-4c6f-b7fe-57577d7f0e0b service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:13.142292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 652/1325] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:15.587095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b352215-8714-4dd8-9a21-ac0aded5af8e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:15.587644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b352215-8714-4dd8-9a21-ac0aded5af8e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:15.587644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b352215-8714-4dd8-9a21-ac0aded5af8e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:15.588921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 674/1326] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:15.594488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ab40a-082f-479f-affb-409b71f948d7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:15.595078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ab40a-082f-479f-affb-409b71f948d7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:15.595078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ab40a-082f-479f-affb-409b71f948d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:15.662103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cf5ab40a-082f-479f-affb-409b71f948d7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:15.663347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cf5ab40a-082f-479f-affb-409b71f948d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:15.766114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 653/1327] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:50:16.051230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b352215-8714-4dd8-9a21-ac0aded5af8e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:16.052900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6b352215-8714-4dd8-9a21-ac0aded5af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:16.123086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:16.173383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b352215-8714-4dd8-9a21-ac0aded5af8e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:16.261815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:16.262426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:16.262709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:16.262982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:16.264640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:16.381925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:16.384812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:16.386721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298db8e-89a3-4e56-a717-b5427209a691 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:16.406184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 675/1328] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:18.675851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb98a90-df1b-4ee7-b717-02109727ba98 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:18.676402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb98a90-df1b-4ee7-b717-02109727ba98 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:18.676402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb98a90-df1b-4ee7-b717-02109727ba98 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:18.677758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 654/1329] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:18.683501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad351-8e98-4e23-bad5-482d4c09fffe None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:18.684016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad351-8e98-4e23-bad5-482d4c09fffe None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:18.684141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fad351-8e98-4e23-bad5-482d4c09fffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:18.755339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-18fad351-8e98-4e23-bad5-482d4c09fffe None None] MFA Rules not processed for user `a775d9fba1794acc91783971d2655e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:18.756560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-18fad351-8e98-4e23-bad5-482d4c09fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:18.854915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 676/1330] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:19.141294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb98a90-df1b-4ee7-b717-02109727ba98 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:19.143048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eeb98a90-df1b-4ee7-b717-02109727ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:19.178407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:19.258383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb98a90-df1b-4ee7-b717-02109727ba98 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:19.338795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:19.339491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:19.339765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:19.340032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:19.341613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:19.419987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:19.453379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:19.454814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d3d09-16f8-439c-ba24-6b3b3756d5ca service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:19.475250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 655/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:22.036514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567278d-2816-4018-add3-e802f60813dd None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:22.037086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567278d-2816-4018-add3-e802f60813dd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:22.037086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567278d-2816-4018-add3-e802f60813dd None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:22.038421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 677/1332] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:22.043662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823df74-4771-470b-a672-bbf8023d1b9e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:22.044172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823df74-4771-470b-a672-bbf8023d1b9e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:22.044281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823df74-4771-470b-a672-bbf8023d1b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:22.109078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5823df74-4771-470b-a672-bbf8023d1b9e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:22.110282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5823df74-4771-470b-a672-bbf8023d1b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:22.208519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 656/1333] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:22.497329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567278d-2816-4018-add3-e802f60813dd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:22.498859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9567278d-2816-4018-add3-e802f60813dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:22.569690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:22.603280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567278d-2816-4018-add3-e802f60813dd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:22.680002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:22.680606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:22.680877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:22.681143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:22.682734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:22.788624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:22.789982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66478847-b7bf-4924-8416-be5642ca697c service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:22.800323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:22.808666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 678/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:50:25.045714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30591a3-be5e-4887-8f6c-4ed75cb816fd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:25.046278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30591a3-be5e-4887-8f6c-4ed75cb816fd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:25.046278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30591a3-be5e-4887-8f6c-4ed75cb816fd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:25.047641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 657/1335] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:25.053113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1d0c-46b1-420d-84aa-14bf0d3b5b0b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:25.053539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1d0c-46b1-420d-84aa-14bf0d3b5b0b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:25.053696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc1d0c-46b1-420d-84aa-14bf0d3b5b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:25.119055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b9bc1d0c-46b1-420d-84aa-14bf0d3b5b0b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:25.120289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b9bc1d0c-46b1-420d-84aa-14bf0d3b5b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:25.216721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 679/1336] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:25.498107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30591a3-be5e-4887-8f6c-4ed75cb816fd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:25.499682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c30591a3-be5e-4887-8f6c-4ed75cb816fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:25.541620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:25.604503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30591a3-be5e-4887-8f6c-4ed75cb816fd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:25.676973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:25.677581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:25.677859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:25.678128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:25.679759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:25.763104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:25.789137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:25.790571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bbda7-f9d8-4fe9-888e-4724adf050b3 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:25.808871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 658/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:28.300734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921eb5e-bcf2-4e27-975e-3a628bffbda9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:28.301342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921eb5e-bcf2-4e27-975e-3a628bffbda9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:28.301342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921eb5e-bcf2-4e27-975e-3a628bffbda9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:28.302637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 680/1338] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:28.307885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d518e-4188-49c7-8896-8a177bebd078 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:28.308293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d518e-4188-49c7-8896-8a177bebd078 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:28.308437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d518e-4188-49c7-8896-8a177bebd078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:28.373476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-062d518e-4188-49c7-8896-8a177bebd078 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:28.374701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-062d518e-4188-49c7-8896-8a177bebd078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:28.469132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 659/1339] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:28.755184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921eb5e-bcf2-4e27-975e-3a628bffbda9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:28.756772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f921eb5e-bcf2-4e27-975e-3a628bffbda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:28.831066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:28.858562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921eb5e-bcf2-4e27-975e-3a628bffbda9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:28.934075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:28.934680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:28.934957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:28.935264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:28.937510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:29.049862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:29.051374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1eeb9-8f4f-49f2-89f5-4848df641689 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:29.069093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 681/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:33.210544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8388b-b89c-4550-a8df-9dc36125dc1d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:33.211074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8388b-b89c-4550-a8df-9dc36125dc1d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:33.211292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8388b-b89c-4550-a8df-9dc36125dc1d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:33.212640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 660/1341] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:33.217983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af885f7f-3354-4c1e-bc97-c4f8b79b4bcf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:33.218435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af885f7f-3354-4c1e-bc97-c4f8b79b4bcf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:33.218542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af885f7f-3354-4c1e-bc97-c4f8b79b4bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:33.224296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:33.286201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-af885f7f-3354-4c1e-bc97-c4f8b79b4bcf None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:33.287343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-af885f7f-3354-4c1e-bc97-c4f8b79b4bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:33.384084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 682/1342] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:33.676901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8388b-b89c-4550-a8df-9dc36125dc1d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:33.678456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-43a8388b-b89c-4550-a8df-9dc36125dc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:33.730612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:33.786921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8388b-b89c-4550-a8df-9dc36125dc1d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:33.861221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:33.862398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:33.862687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:33.862963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:33.864930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:33.961704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:33.979626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:33.981166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c54b-c910-4fcc-b8c6-e2f74f1188c8 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:34.011040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 661/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:36.874977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48c4dd-a76d-440d-8303-6e3f43b1279c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:36.875619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48c4dd-a76d-440d-8303-6e3f43b1279c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:36.875619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48c4dd-a76d-440d-8303-6e3f43b1279c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:36.876932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 683/1344] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:36.883047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8f18-e9c5-480a-98e6-75f83e92161c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:36.883591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8f18-e9c5-480a-98e6-75f83e92161c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:36.883676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8f18-e9c5-480a-98e6-75f83e92161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:36.950184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-587d8f18-e9c5-480a-98e6-75f83e92161c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:36.951443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-587d8f18-e9c5-480a-98e6-75f83e92161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:37.047489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 662/1345] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:37.339102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48c4dd-a76d-440d-8303-6e3f43b1279c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:37.340768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ac48c4dd-a76d-440d-8303-6e3f43b1279c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:37.433375 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:37.451306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48c4dd-a76d-440d-8303-6e3f43b1279c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:37.529216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:37.529815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:37.530090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:37.530352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:37.531952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:37.646219 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:37.647703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27979017-3c14-46fa-92db-2cb7c54af392 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:37.666282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 684/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:41.875592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659873cc-b274-4b64-baf6-fffa96aef322 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:41.876061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659873cc-b274-4b64-baf6-fffa96aef322 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:41.876143 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659873cc-b274-4b64-baf6-fffa96aef322 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:41.878508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 663/1347] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:41.884811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d48ac3-4c32-4aa1-8924-fd82321a6412 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:41.885689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d48ac3-4c32-4aa1-8924-fd82321a6412 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:41.886120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d48ac3-4c32-4aa1-8924-fd82321a6412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:41.903128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:41.965143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-04d48ac3-4c32-4aa1-8924-fd82321a6412 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:41.966365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-04d48ac3-4c32-4aa1-8924-fd82321a6412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:42.098363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 685/1348] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:42.380101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659873cc-b274-4b64-baf6-fffa96aef322 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:42.381689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-659873cc-b274-4b64-baf6-fffa96aef322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:42.438247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:42.486994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659873cc-b274-4b64-baf6-fffa96aef322 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:42.561246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:42.561844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:42.562123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:42.562393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:42.564010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:42.672451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:42.677229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:42.678726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04dc3e-c700-4d6b-8fa8-5e424632d201 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:42.698919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 664/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:47.901653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b5032-627b-49dd-8812-88daada7fd7e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:47.902194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b5032-627b-49dd-8812-88daada7fd7e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:47.902194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b5032-627b-49dd-8812-88daada7fd7e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:47.904654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 686/1350] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:47.910612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed71d7-4f11-4f72-9984-addb07cf90ee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:47.912162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed71d7-4f11-4f72-9984-addb07cf90ee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:47.912162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed71d7-4f11-4f72-9984-addb07cf90ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:48.044721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a4ed71d7-4f11-4f72-9984-addb07cf90ee None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:48.046320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a4ed71d7-4f11-4f72-9984-addb07cf90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:48.228860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 665/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:50:52.098508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=59 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:53.306654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f544c4-9fbe-4c27-8d57-4be59259cb2c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:53.307335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f544c4-9fbe-4c27-8d57-4be59259cb2c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:53.307335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f544c4-9fbe-4c27-8d57-4be59259cb2c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:53.308584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 687/1352] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:53.313904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a8908-7c5c-41b3-86af-71f76468c08d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:53.315321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a8908-7c5c-41b3-86af-71f76468c08d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:53.315812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a8908-7c5c-41b3-86af-71f76468c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:53.373339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:53.400487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cd4a8908-7c5c-41b3-86af-71f76468c08d None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:53.402233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cd4a8908-7c5c-41b3-86af-71f76468c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:53.526938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 666/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:50:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:55.024928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26736e97-6546-41ee-8cba-eefaa506805a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:55.025492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26736e97-6546-41ee-8cba-eefaa506805a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:55.025492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26736e97-6546-41ee-8cba-eefaa506805a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:55.026822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 688/1354] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:55.032209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e25032-9712-45a5-97cf-eb198e702604 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:55.032702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e25032-9712-45a5-97cf-eb198e702604 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:55.032800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e25032-9712-45a5-97cf-eb198e702604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:55.114761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-41e25032-9712-45a5-97cf-eb198e702604 None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:55.116001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-41e25032-9712-45a5-97cf-eb198e702604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:55.164806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:55.249696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 667/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:50:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:58.906454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac64a-a23e-46a3-8c47-cbf353e37a2a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:58.907661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac64a-a23e-46a3-8c47-cbf353e37a2a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:58.907661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac64a-a23e-46a3-8c47-cbf353e37a2a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:58.910632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 689/1356] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:50:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:58.919833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7a0a-6f90-4467-86de-ec9b520ecfd3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:58.920367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7a0a-6f90-4467-86de-ec9b520ecfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:58.920792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e7a0a-6f90-4467-86de-ec9b520ecfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:59.028983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-814e7a0a-6f90-4467-86de-ec9b520ecfd3 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:59.030898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-814e7a0a-6f90-4467-86de-ec9b520ecfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:59.192745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 668/1357] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:59.781622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833746d5-99ba-410d-aca7-c5bcd5420aa2 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:59.782606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833746d5-99ba-410d-aca7-c5bcd5420aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:59.783507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833746d5-99ba-410d-aca7-c5bcd5420aa2 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:59.786552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 690/1358] 10.222.0.66 () {54 vars in 961 bytes} [Thu Sep 22 14:50:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:59.798153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75eed-c87c-4fd7-8e48-5aedf9511854 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:59.799564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75eed-c87c-4fd7-8e48-5aedf9511854 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:59.800888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75eed-c87c-4fd7-8e48-5aedf9511854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:59.831809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:59.962261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2bc75eed-c87c-4fd7-8e48-5aedf9511854 None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:59.963843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2bc75eed-c87c-4fd7-8e48-5aedf9511854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:00.146677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 669/1359] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Sep 22 14:50:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:00.156130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833746d5-99ba-410d-aca7-c5bcd5420aa2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:00.159263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-833746d5-99ba-410d-aca7-c5bcd5420aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:00.386386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833746d5-99ba-410d-aca7-c5bcd5420aa2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:00.504630 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:00.590389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:00.591864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:00.592515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:00.593121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:00.597112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:00.755698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:00.757662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defed82a-94da-46ab-916c-afd714d4eb2b service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:00.787970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 691/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:03.341746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9cfc7-6a2f-4861-91df-abad4e801d1a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:03.342550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9cfc7-6a2f-4861-91df-abad4e801d1a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:03.343327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9cfc7-6a2f-4861-91df-abad4e801d1a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:03.346414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 670/1361] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:03.352594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b032-f251-4a1f-92a0-f5ca7ded148b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:03.353046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b032-f251-4a1f-92a0-f5ca7ded148b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:03.353207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b032-f251-4a1f-92a0-f5ca7ded148b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:03.434985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5eb0b032-f251-4a1f-92a0-f5ca7ded148b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:03.436417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5eb0b032-f251-4a1f-92a0-f5ca7ded148b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:03.491071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:03.575476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 692/1362] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:03.888244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9cfc7-6a2f-4861-91df-abad4e801d1a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:03.889680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-77f9cfc7-6a2f-4861-91df-abad4e801d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:03.964936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:04.093632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9cfc7-6a2f-4861-91df-abad4e801d1a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:04.225758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:04.226380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:04.226656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:04.226925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:04.228591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:04.264295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:04.356454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:04.358364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485c2ed-943e-428b-83a2-a60e2ae3bea6 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:04.382503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 671/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:07.005541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c1757-4eea-48fe-8332-d0eb907900e9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.006137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c1757-4eea-48fe-8332-d0eb907900e9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.006137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c1757-4eea-48fe-8332-d0eb907900e9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.007704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 693/1364] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:07.013297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca24ed-10d3-4b2b-9f42-f23a9039f2be None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.013736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca24ed-10d3-4b2b-9f42-f23a9039f2be None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.013940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca24ed-10d3-4b2b-9f42-f23a9039f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.085629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-20ca24ed-10d3-4b2b-9f42-f23a9039f2be None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:07.086840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-20ca24ed-10d3-4b2b-9f42-f23a9039f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:07.192667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:07.195082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 672/1365] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:07.495568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e6c77-1a9c-410a-9b50-d9d24fa65757 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.495988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e6c77-1a9c-410a-9b50-d9d24fa65757 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.496496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e6c77-1a9c-410a-9b50-d9d24fa65757 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.498271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 694/1366] 10.222.0.66 () {54 vars in 961 bytes} [Thu Sep 22 14:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:07.504725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.505018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.505301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.595608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:07.596867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:07.652211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314006e-6614-4afd-87ed-764ef1344367 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.652498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314006e-6614-4afd-87ed-764ef1344367 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.652782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314006e-6614-4afd-87ed-764ef1344367 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.654124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 695/1367] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:07.659176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e47cfa-0f1d-4342-91b3-4808e3d3a805 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.659655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e47cfa-0f1d-4342-91b3-4808e3d3a805 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.659780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e47cfa-0f1d-4342-91b3-4808e3d3a805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.726525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 673/1368] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Sep 22 14:51:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:07.729784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-96e47cfa-0f1d-4342-91b3-4808e3d3a805 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:07.731382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96e47cfa-0f1d-4342-91b3-4808e3d3a805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:07.734894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:07.745672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dae76-1887-429b-a75b-43e9b5b1a891 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:07.803451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:07.852096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 696/1369] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:07.909475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:07.910202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:07.910549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:07.910890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:07.912623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:08.044758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:08.048168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c34837-37c7-4160-b238-90dbfae9afa2 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:08.089534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 674/1370] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:51:08.163116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3516116-924e-405b-a8fe-2a8e437115c6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:08.163739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3516116-924e-405b-a8fe-2a8e437115c6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:08.164372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3516116-924e-405b-a8fe-2a8e437115c6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:08.167263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 697/1371] 10.222.0.66 () {54 vars in 963 bytes} [Thu Sep 22 14:51:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:08.178714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4eea-b79c-4f8e-b018-402ae1de701d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:08.179760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4eea-b79c-4f8e-b018-402ae1de701d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:08.180247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4eea-b79c-4f8e-b018-402ae1de701d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:08.278803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:08.304596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-979d4eea-b79c-4f8e-b018-402ae1de701d None None] MFA Rules not processed for user `b4e09c957a104f2299ab55d8283124c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:08.305929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-979d4eea-b79c-4f8e-b018-402ae1de701d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:08.405609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 675/1372] 10.222.0.66 () {58 vars in 1060 bytes} [Thu Sep 22 14:51:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:08.413942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3516116-924e-405b-a8fe-2a8e437115c6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:08.415663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d3516116-924e-405b-a8fe-2a8e437115c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:08.447435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:08.547679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3516116-924e-405b-a8fe-2a8e437115c6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:08.645768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:08.646394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:08.646674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:08.647016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:08.648625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:08.740850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:08.791124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:08.792659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b4f86-2f6a-4082-9465-104963011929 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:08.813659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 698/1373] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:10.512819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da7d3-4d29-4d06-80e9-a13400709361 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:10.513375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da7d3-4d29-4d06-80e9-a13400709361 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:10.513375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da7d3-4d29-4d06-80e9-a13400709361 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:10.514787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 676/1374] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:10.522017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229bc12-650e-4e34-adfc-68171c920ea4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:10.522578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229bc12-650e-4e34-adfc-68171c920ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:10.522578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229bc12-650e-4e34-adfc-68171c920ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:10.598983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f229bc12-650e-4e34-adfc-68171c920ea4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:10.600263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f229bc12-650e-4e34-adfc-68171c920ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:10.719710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 699/1375] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:11.007444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da7d3-4d29-4d06-80e9-a13400709361 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:11.009281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c27da7d3-4d29-4d06-80e9-a13400709361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:11.164010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:11.169889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da7d3-4d29-4d06-80e9-a13400709361 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:11.299288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:11.299834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:11.300114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:11.300393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:11.302030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:11.407030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:11.410844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650961-235b-41a6-b536-abcb4ca48dab service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:11.428215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:11.435540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 677/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:13.720785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d7f5-ca7b-43c1-bfeb-fe62e1f3d141 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:13.721324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d7f5-ca7b-43c1-bfeb-fe62e1f3d141 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:13.721406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d7f5-ca7b-43c1-bfeb-fe62e1f3d141 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:13.722831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 700/1377] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:13.728701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1332704-96cd-4184-89c0-c35008983a6e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:13.729210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1332704-96cd-4184-89c0-c35008983a6e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:13.729210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1332704-96cd-4184-89c0-c35008983a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:13.819087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a1332704-96cd-4184-89c0-c35008983a6e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:13.820554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a1332704-96cd-4184-89c0-c35008983a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:13.941477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 678/1378] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:14.249917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d7f5-ca7b-43c1-bfeb-fe62e1f3d141 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:14.253730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c948d7f5-ca7b-43c1-bfeb-fe62e1f3d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:14.305903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:14.480717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d7f5-ca7b-43c1-bfeb-fe62e1f3d141 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:14.596767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:14.612349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:14.613017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:14.613293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:14.613561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:14.615381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:14.755929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:14.757581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4fa1b-0ede-438b-a6d8-a1fb910255a7 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:14.786802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 701/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:17.072541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155f5ca-01b0-4f55-8992-c7c5e7001a3b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:17.073221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155f5ca-01b0-4f55-8992-c7c5e7001a3b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:17.073221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155f5ca-01b0-4f55-8992-c7c5e7001a3b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:17.074419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 679/1380] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:17.080207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cbba6-b188-401b-bbf1-3a8616d084eb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:17.080654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cbba6-b188-401b-bbf1-3a8616d084eb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:17.080774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cbba6-b188-401b-bbf1-3a8616d084eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:17.153350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a92cbba6-b188-401b-bbf1-3a8616d084eb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:17.154763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a92cbba6-b188-401b-bbf1-3a8616d084eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:17.224541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:17.262812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 702/1381] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:17.554069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155f5ca-01b0-4f55-8992-c7c5e7001a3b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:17.555694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6155f5ca-01b0-4f55-8992-c7c5e7001a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:17.653987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:17.723597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155f5ca-01b0-4f55-8992-c7c5e7001a3b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:17.838506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:17.839148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:17.839409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:17.839671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:17.841254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:17.889815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:17.944214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:17.946017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4795b6-f76e-4d10-884c-8b1bb2575443 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:17.968119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 680/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:20.434862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677182cd-7640-4f1e-b12d-d8e1f4338867 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:20.435642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677182cd-7640-4f1e-b12d-d8e1f4338867 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:20.435642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677182cd-7640-4f1e-b12d-d8e1f4338867 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:20.436796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 703/1383] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:20.442097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06048297-d0f4-42e5-83db-b2e3ca8ea023 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:20.442560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06048297-d0f4-42e5-83db-b2e3ca8ea023 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:20.442663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06048297-d0f4-42e5-83db-b2e3ca8ea023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:20.514844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-06048297-d0f4-42e5-83db-b2e3ca8ea023 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:20.516139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-06048297-d0f4-42e5-83db-b2e3ca8ea023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:20.620515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 681/1384] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:20.907166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677182cd-7640-4f1e-b12d-d8e1f4338867 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:20.909310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-677182cd-7640-4f1e-b12d-d8e1f4338867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.091799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677182cd-7640-4f1e-b12d-d8e1f4338867 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:21.150677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:21.219031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589eb580-7178-458a-812c-230d997e4066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:21.219586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eb580-7178-458a-812c-230d997e4066 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:21.219860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eb580-7178-458a-812c-230d997e4066 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:21.220142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589eb580-7178-458a-812c-230d997e4066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:21.221742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-589eb580-7178-458a-812c-230d997e4066 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.333563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eb580-7178-458a-812c-230d997e4066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:21.335234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589eb580-7178-458a-812c-230d997e4066 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:21.354902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 704/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:23.580260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746443-ab18-4ffd-8cb3-5f61476478c4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:23.580841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746443-ab18-4ffd-8cb3-5f61476478c4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:23.580841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746443-ab18-4ffd-8cb3-5f61476478c4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:23.582669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 682/1386] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:23.589002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0a700-5774-45a6-8b9f-4fbf15984739 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:23.589516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0a700-5774-45a6-8b9f-4fbf15984739 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:23.589633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0a700-5774-45a6-8b9f-4fbf15984739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:23.662160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d9a0a700-5774-45a6-8b9f-4fbf15984739 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:23.664089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d9a0a700-5774-45a6-8b9f-4fbf15984739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:23.668666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:23.781427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 705/1387] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:24.073262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746443-ab18-4ffd-8cb3-5f61476478c4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:24.074939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-33746443-ab18-4ffd-8cb3-5f61476478c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:24.095423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:24.226037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746443-ab18-4ffd-8cb3-5f61476478c4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:24.322618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:24.346622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:24.347249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:24.347535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:24.347810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:24.349429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:24.451624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:24.453226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c63c6-8a41-45b8-8842-16c39288433c service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:24.474098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 683/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:51:24.844279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef7172-bc7e-4641-96ff-77e554bf12e3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:24.844845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef7172-bc7e-4641-96ff-77e554bf12e3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:24.844845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef7172-bc7e-4641-96ff-77e554bf12e3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:24.846888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 706/1389] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:24.852262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4dfb-b16f-41b6-9119-cf450f774008 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:24.852726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4dfb-b16f-41b6-9119-cf450f774008 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:24.852816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4dfb-b16f-41b6-9119-cf450f774008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:24.921336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9a4b4dfb-b16f-41b6-9119-cf450f774008 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:24.922461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9a4b4dfb-b16f-41b6-9119-cf450f774008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:24.985786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:25.037554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 684/1390] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:25.341473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de9a71-a01c-4da2-b86c-e4ce3b749924 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:25.341762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de9a71-a01c-4da2-b86c-e4ce3b749924 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:25.342038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de9a71-a01c-4da2-b86c-e4ce3b749924 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:25.343421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 707/1391] 10.222.0.66 () {54 vars in 962 bytes} [Thu Sep 22 14:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:25.349433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1558a-3a56-42af-8426-a4ff689f08e0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:25.349808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1558a-3a56-42af-8426-a4ff689f08e0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:25.350145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1558a-3a56-42af-8426-a4ff689f08e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:25.429070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c6a1558a-3a56-42af-8426-a4ff689f08e0 None None] MFA Rules not processed for user `623f9046c3924b35b36d8807f8eef54c`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:25.430273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c6a1558a-3a56-42af-8426-a4ff689f08e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:25.512468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 685/1392] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Sep 22 14:51:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:25.523907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de9a71-a01c-4da2-b86c-e4ce3b749924 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:25.525898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f3de9a71-a01c-4da2-b86c-e4ce3b749924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:25.634823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de9a71-a01c-4da2-b86c-e4ce3b749924 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:25.668747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:25.712549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:25.713159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:25.713444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:25.713812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:25.715441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:25.830107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:25.831594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f746d2-e954-43b1-97fc-c4ef63ee9037 service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:25.849669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 708/1393] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:26.611081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b4842-b869-4f6c-9fa2-a7b217df97f0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:26.611688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b4842-b869-4f6c-9fa2-a7b217df97f0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:26.611688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b4842-b869-4f6c-9fa2-a7b217df97f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:26.681394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:26.684785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-034b4842-b869-4f6c-9fa2-a7b217df97f0 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:26.686398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-034b4842-b869-4f6c-9fa2-a7b217df97f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:26.708641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e68d4-abbe-450a-8ecd-7292c81109e8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:26.708945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e68d4-abbe-450a-8ecd-7292c81109e8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:26.709199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e68d4-abbe-450a-8ecd-7292c81109e8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:26.710742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 709/1394] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:51:26.716529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:26.716811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:26.717084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:26.776868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 686/1395] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:26.784960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:26.786637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:26.800921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:26.824584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd3697-9398-48d8-902e-ecd2ce881931 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:26.824954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd3697-9398-48d8-902e-ecd2ce881931 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:26.825296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd3697-9398-48d8-902e-ecd2ce881931 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:26.827358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 687/1396] 10.222.0.66 () {54 vars in 943 bytes} [Thu Sep 22 14:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:51:26.833714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:26.833997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:26.834271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:26.888961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 710/1397] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:26.915660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] MFA Rules not processed for user `543542d17db0495f8151e779e6c6218f`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:26.916927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:26.999518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:27.033833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 688/1398] 10.222.0.66 () {58 vars in 1040 bytes} [Thu Sep 22 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:27.043324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:27.044556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:27.152775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c67f83-83f3-41eb-b369-341d2b85ef29 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:27.184323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:27.185549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:27.196803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:27.264230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:27.264843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:27.265121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:27.265395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:27.267619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:27.288735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e755e1f-8fa5-49ee-91f8-a6404e0dc14e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:27.356951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:27.358592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae36e42-41ef-4c26-8f8e-4b5004d4f1d8 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:27.367476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 711/1399] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 14:51:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 14:51:27.392038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:27.412953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:27.413562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:27.413844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:27.414113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:27.415749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:27.528964 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:27.530968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffaf1f-6e54-404f-b3a8-ff78cf276b85 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:27.560277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 689/1400] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:29.789435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713609-a4e6-4770-8c14-1f03aac06008 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:29.790066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713609-a4e6-4770-8c14-1f03aac06008 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:29.790066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713609-a4e6-4770-8c14-1f03aac06008 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:29.795250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 712/1401] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:29.796906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38dcbf-986c-4e28-b3b6-42b241d6b33f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:29.797366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38dcbf-986c-4e28-b3b6-42b241d6b33f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:29.797493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38dcbf-986c-4e28-b3b6-42b241d6b33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:29.873449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8a38dcbf-986c-4e28-b3b6-42b241d6b33f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:29.874847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8a38dcbf-986c-4e28-b3b6-42b241d6b33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:29.947177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:30.011042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 690/1402] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:30.301762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713609-a4e6-4770-8c14-1f03aac06008 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:30.303353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-28713609-a4e6-4770-8c14-1f03aac06008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:30.389574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:30.465563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713609-a4e6-4770-8c14-1f03aac06008 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:30.588869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:30.589611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:30.589966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:30.590303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:30.592168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:30.639569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:30.703696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:30.705331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47af74-b1bf-4fd1-b9b6-6545feb8c3f1 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:30.725108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 713/1403] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:32.943187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd45d5-6dbc-4c02-8f2f-449a799fd0f2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:32.943901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd45d5-6dbc-4c02-8f2f-449a799fd0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:32.943901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd45d5-6dbc-4c02-8f2f-449a799fd0f2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:32.945118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 691/1404] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:32.951605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8e37c-cfb6-4cb8-940b-4a17c0433415 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:32.952075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8e37c-cfb6-4cb8-940b-4a17c0433415 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:32.952189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8e37c-cfb6-4cb8-940b-4a17c0433415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:33.026387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-49a8e37c-cfb6-4cb8-940b-4a17c0433415 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:33.027805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-49a8e37c-cfb6-4cb8-940b-4a17c0433415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:33.139756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 714/1405] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:33.429280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd45d5-6dbc-4c02-8f2f-449a799fd0f2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:33.431067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5fcd45d5-6dbc-4c02-8f2f-449a799fd0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:33.618872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd45d5-6dbc-4c02-8f2f-449a799fd0f2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:33.673405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:33.756416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:33.757207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:33.757554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:33.757950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:33.760080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:33.861749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:33.863427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10c5ad-41e0-481b-aa92-bb906d0660c7 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:33.883281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 692/1406] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:36.162405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871e3ef-c329-492a-bb15-2901c92a169e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:36.163008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871e3ef-c329-492a-bb15-2901c92a169e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:36.163008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871e3ef-c329-492a-bb15-2901c92a169e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:36.164340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 715/1407] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:36.169699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c5f4f-3b10-4ab5-b6c3-1885f05a2191 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:36.170106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c5f4f-3b10-4ab5-b6c3-1885f05a2191 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:36.170304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c5f4f-3b10-4ab5-b6c3-1885f05a2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:36.240243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:36.249264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-433c5f4f-3b10-4ab5-b6c3-1885f05a2191 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:36.250655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-433c5f4f-3b10-4ab5-b6c3-1885f05a2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:36.362171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 693/1408] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:36.663177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871e3ef-c329-492a-bb15-2901c92a169e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:36.664818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2871e3ef-c329-492a-bb15-2901c92a169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:36.678644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:36.833451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871e3ef-c329-492a-bb15-2901c92a169e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:36.927434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:36.965404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:36.966343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:36.966620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:36.966891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:36.968538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:37.068520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:37.070294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbc016-c512-487f-9519-c55ff974aafd service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:37.094435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 716/1409] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:46.360120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=44 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:47.089337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=57 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:50.427735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6f1a7-3858-4ae0-b0ab-ddf27e91c79b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.428351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6f1a7-3858-4ae0-b0ab-ddf27e91c79b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.428351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6f1a7-3858-4ae0-b0ab-ddf27e91c79b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.429661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 694/1410] 10.222.0.66 () {54 vars in 918 bytes} [Thu Sep 22 14:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:50.434859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e56a3-8f45-48b2-b35e-4a6c4ccac6bf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.435325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e56a3-8f45-48b2-b35e-4a6c4ccac6bf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.435440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e56a3-8f45-48b2-b35e-4a6c4ccac6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.512790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3b1e56a3-8f45-48b2-b35e-4a6c4ccac6bf None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:50.513999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3b1e56a3-8f45-48b2-b35e-4a6c4ccac6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:50.630067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 717/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Sep 22 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:50.645131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7cb7-229e-4075-bed2-7b7e40130517 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.645423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7cb7-229e-4075-bed2-7b7e40130517 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.645699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe7cb7-229e-4075-bed2-7b7e40130517 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.647035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 695/1412] 10.222.0.66 () {54 vars in 918 bytes} [Thu Sep 22 14:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:50.652668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146af81-7541-4dc2-aa33-e71f4be3526f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.652668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146af81-7541-4dc2-aa33-e71f4be3526f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.652957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146af81-7541-4dc2-aa33-e71f4be3526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.733034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:50.736212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6146af81-7541-4dc2-aa33-e71f4be3526f None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:50.737967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6146af81-7541-4dc2-aa33-e71f4be3526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:50.851819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 718/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Sep 22 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:53.465401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bcf51-37b2-4102-9b4e-f9ddacb3bae2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:53.465919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bcf51-37b2-4102-9b4e-f9ddacb3bae2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:53.465919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bcf51-37b2-4102-9b4e-f9ddacb3bae2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:53.467188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:53.472901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa9582-3aa1-4137-8ea5-f59bc3d853f3 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:53.473379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa9582-3aa1-4137-8ea5-f59bc3d853f3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:53.473481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa9582-3aa1-4137-8ea5-f59bc3d853f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:53.540556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e3fa9582-3aa1-4137-8ea5-f59bc3d853f3 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:53.541772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e3fa9582-3aa1-4137-8ea5-f59bc3d853f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:53.599284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:53.642906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:53.932857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bcf51-37b2-4102-9b4e-f9ddacb3bae2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:53.934518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1d1bcf51-37b2-4102-9b4e-f9ddacb3bae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:54.083581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bcf51-37b2-4102-9b4e-f9ddacb3bae2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:54.159280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:54.201072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:54.201669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:54.201941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:54.202371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:54.203878 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:54.296277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:54.298498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2dc14-7a8b-43c6-8617-1292748929fe service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:54.316095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 697/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:55.462326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013810f1-3940-44b8-b894-d573c4c60336 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:55.462907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013810f1-3940-44b8-b894-d573c4c60336 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:55.462907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013810f1-3940-44b8-b894-d573c4c60336 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:55.464239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:55.471308 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a128c-f27b-48c5-a5f5-c6d465d881ca None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:55.471766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a128c-f27b-48c5-a5f5-c6d465d881ca None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:55.471865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a128c-f27b-48c5-a5f5-c6d465d881ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:55.560300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5e8a128c-f27b-48c5-a5f5-c6d465d881ca None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:55.561528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5e8a128c-f27b-48c5-a5f5-c6d465d881ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:55.587822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:55.658786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:55.811757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013810f1-3940-44b8-b894-d573c4c60336 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:55.813411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-013810f1-3940-44b8-b894-d573c4c60336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:55.966382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013810f1-3940-44b8-b894-d573c4c60336 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:56.006490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:56.080917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:56.081530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:56.081809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:56.082080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:56.083714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:56.173787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:56.175418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671e3b2-4e5d-4cc2-8994-0882601e6b96 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:56.193126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 721/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:57.328364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fdc4f-18f7-429e-8435-a08b14be6f64 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:57.329533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fdc4f-18f7-429e-8435-a08b14be6f64 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:57.329533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fdc4f-18f7-429e-8435-a08b14be6f64 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:57.330733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:57.336991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b49870-ce93-45fd-a43e-f3ebaf33dcb2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:57.337463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b49870-ce93-45fd-a43e-f3ebaf33dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:57.337580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b49870-ce93-45fd-a43e-f3ebaf33dcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:57.392219 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:57.415846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-52b49870-ce93-45fd-a43e-f3ebaf33dcb2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:57.417054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-52b49870-ce93-45fd-a43e-f3ebaf33dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:57.514032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:57.665788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fdc4f-18f7-429e-8435-a08b14be6f64 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:57.667262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-884fdc4f-18f7-429e-8435-a08b14be6f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:57.816083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fdc4f-18f7-429e-8435-a08b14be6f64 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:57.827925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:57.938160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:57.938679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:57.938958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:57.939252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:57.940858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:58.031643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:58.033782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2abf8-791a-4d01-a286-77ea4c2e7329 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:58.052627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 700/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:51:59.176143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542796-e89c-4590-a6f2-dafb9e37b35e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:59.176706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542796-e89c-4590-a6f2-dafb9e37b35e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:59.176706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b542796-e89c-4590-a6f2-dafb9e37b35e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:59.178031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:59.184562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bcedf-eb67-4f15-94af-0b6ec44fb3d4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:59.185061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bcedf-eb67-4f15-94af-0b6ec44fb3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:59.185165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bcedf-eb67-4f15-94af-0b6ec44fb3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:59.205084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:59.264298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-009bcedf-eb67-4f15-94af-0b6ec44fb3d4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:59.265505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-009bcedf-eb67-4f15-94af-0b6ec44fb3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:59.359399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:59.512342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b542796-e89c-4590-a6f2-dafb9e37b35e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:59.513908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4b542796-e89c-4590-a6f2-dafb9e37b35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:59.623121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:59.660438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b542796-e89c-4590-a6f2-dafb9e37b35e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:59.775454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:59.776216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:59.776501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:59.776767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:59.778451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:59.855961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:59.869367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:59.870978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d49b9-2cd5-4b1d-8006-f074f240a94b service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:59.889783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 724/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:01.008495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf266c3-9c88-47c4-9ce3-ef090bc31e75 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:01.009306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf266c3-9c88-47c4-9ce3-ef090bc31e75 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:01.009306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf266c3-9c88-47c4-9ce3-ef090bc31e75 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:01.010602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:01.016892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2afe-09db-4b25-ab4f-ea70b2c2a2f1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:01.017430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2afe-09db-4b25-ab4f-ea70b2c2a2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:01.017430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2afe-09db-4b25-ab4f-ea70b2c2a2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:01.093524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5a3b2afe-09db-4b25-ab4f-ea70b2c2a2f1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:01.095347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5a3b2afe-09db-4b25-ab4f-ea70b2c2a2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:01.191543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:01.341627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf266c3-9c88-47c4-9ce3-ef090bc31e75 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:01.343137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ccf266c3-9c88-47c4-9ce3-ef090bc31e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:01.426616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:01.491717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf266c3-9c88-47c4-9ce3-ef090bc31e75 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:01.605833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:01.606369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:01.606648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:01.606936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:01.608581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:01.658478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:01.700050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:01.701657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835ed20-ce87-4552-ab19-82f500c053a9 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:01.720151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 703/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:02.784344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a236d-580c-497f-863a-d2fcb7492328 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:02.784879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a236d-580c-497f-863a-d2fcb7492328 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:02.784879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a236d-580c-497f-863a-d2fcb7492328 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:02.786155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:02.792536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fc36f-f524-4243-9bde-05b61ae0e4bf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:02.793000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fc36f-f524-4243-9bde-05b61ae0e4bf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:02.793115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fc36f-f524-4243-9bde-05b61ae0e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:02.869273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-871fc36f-f524-4243-9bde-05b61ae0e4bf None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:02.870508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-871fc36f-f524-4243-9bde-05b61ae0e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:02.969757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:03.120143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a236d-580c-497f-863a-d2fcb7492328 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:03.121649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-234a236d-580c-497f-863a-d2fcb7492328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:03.170001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:03.275866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a236d-580c-497f-863a-d2fcb7492328 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:03.387974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:03.388494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:03.388767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:03.389036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:03.390623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:03.398011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:03.480275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:03.481902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c3f7f-0579-46a3-917b-c9a9641ce31e service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:03.499801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 727/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:04.603519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc382f-ffa2-43cd-b707-f0c261eca959 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:04.604089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc382f-ffa2-43cd-b707-f0c261eca959 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:04.604089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc382f-ffa2-43cd-b707-f0c261eca959 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:04.605415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:04.611858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc678e-a151-4cd8-aaec-2f3941997ea2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:04.612295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc678e-a151-4cd8-aaec-2f3941997ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:04.612399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc678e-a151-4cd8-aaec-2f3941997ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:04.688692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c9dc678e-a151-4cd8-aaec-2f3941997ea2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:04.689910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c9dc678e-a151-4cd8-aaec-2f3941997ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:04.756639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:04.786844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:04.935521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc382f-ffa2-43cd-b707-f0c261eca959 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:04.937027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-48bc382f-ffa2-43cd-b707-f0c261eca959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:04.955432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:05.081514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc382f-ffa2-43cd-b707-f0c261eca959 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:05.169518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:05.196461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:05.197070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:05.197349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:05.197621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:05.199264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:05.288096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:05.289698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ece04b-e5db-4925-85ca-260b74b18451 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:05.308187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 706/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:06.382735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a705911-d590-4ba7-8d98-b3cab64645c5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:06.383301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a705911-d590-4ba7-8d98-b3cab64645c5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:06.383619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a705911-d590-4ba7-8d98-b3cab64645c5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:06.385371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:06.392660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd618-f45d-4114-944f-6e6372efeeea None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:06.393108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd618-f45d-4114-944f-6e6372efeeea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:06.393231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd618-f45d-4114-944f-6e6372efeeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:06.467334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4e0fd618-f45d-4114-944f-6e6372efeeea None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:06.468564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4e0fd618-f45d-4114-944f-6e6372efeeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:06.510746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:06.565681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:06.717261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a705911-d590-4ba7-8d98-b3cab64645c5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:06.718805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0a705911-d590-4ba7-8d98-b3cab64645c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:06.873207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a705911-d590-4ba7-8d98-b3cab64645c5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:06.931804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:06.992540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:06.993065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:06.993341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:06.993626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:06.995223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:07.085770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:07.087437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bf990-2d16-4d20-bead-e650d213bae0 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:07.105203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 730/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:08.169786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012c5b9-2544-46c2-ba85-40b735562810 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:08.170369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012c5b9-2544-46c2-ba85-40b735562810 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:08.170369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012c5b9-2544-46c2-ba85-40b735562810 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:08.171772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:08.178124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe41a8-b138-4501-aa0d-9a54a45ed518 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:08.178576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe41a8-b138-4501-aa0d-9a54a45ed518 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:08.178673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe41a8-b138-4501-aa0d-9a54a45ed518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:08.257403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f0fe41a8-b138-4501-aa0d-9a54a45ed518 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:08.258676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f0fe41a8-b138-4501-aa0d-9a54a45ed518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:08.261276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:08.364248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:08.512290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012c5b9-2544-46c2-ba85-40b735562810 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:08.513787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0012c5b9-2544-46c2-ba85-40b735562810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:08.664389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012c5b9-2544-46c2-ba85-40b735562810 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:08.693319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:08.779163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:08.779780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:08.780090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:08.780358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:08.782001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:08.871765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:08.873359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae94d6b9-033a-4773-a5a7-27263eab2559 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:08.892474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 709/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:10.977525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdde40-313a-4c14-ba49-ba83b2a65f59 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:10.978178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdde40-313a-4c14-ba49-ba83b2a65f59 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:10.978178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdde40-313a-4c14-ba49-ba83b2a65f59 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:10.979742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:10.984909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd01ee-cff4-4b0d-b41f-9430d2d60815 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:10.985328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd01ee-cff4-4b0d-b41f-9430d2d60815 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:10.985440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd01ee-cff4-4b0d-b41f-9430d2d60815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:11.028330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:11.054131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-42fd01ee-cff4-4b0d-b41f-9430d2d60815 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:11.055365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-42fd01ee-cff4-4b0d-b41f-9430d2d60815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:11.150211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:11.437843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdde40-313a-4c14-ba49-ba83b2a65f59 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:11.439412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fdfdde40-313a-4c14-ba49-ba83b2a65f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:11.542882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdde40-313a-4c14-ba49-ba83b2a65f59 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:11.574752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:11.616695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:11.617306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:11.617577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:11.617846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:11.619465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:11.728728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:11.730096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439f09e-5883-4854-a28d-51b1bf425b17 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:11.748350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 733/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:14.110649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82416f-1103-472c-8cbf-eefe5eca32cb None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:14.111241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82416f-1103-472c-8cbf-eefe5eca32cb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:14.111241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82416f-1103-472c-8cbf-eefe5eca32cb None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:14.112600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:14.117891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d37a3-c602-4b5b-ae30-ca7d8422ef19 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:14.118343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d37a3-c602-4b5b-ae30-ca7d8422ef19 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:14.118457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d37a3-c602-4b5b-ae30-ca7d8422ef19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:14.184636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1e1d37a3-c602-4b5b-ae30-ca7d8422ef19 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:14.185855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1e1d37a3-c602-4b5b-ae30-ca7d8422ef19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:14.198462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:14.288873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:14.573757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82416f-1103-472c-8cbf-eefe5eca32cb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:14.575342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dc82416f-1103-472c-8cbf-eefe5eca32cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:14.685410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82416f-1103-472c-8cbf-eefe5eca32cb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:14.700967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:14.764305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:14.764829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:14.765110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:14.765383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:14.767014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:14.998721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:15.000199 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe82bc-33a4-441d-b676-1b04c5c3211b service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:15.018561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 712/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:17.593673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690d02e-36c1-46dd-a979-f38a4372a573 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:17.594253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690d02e-36c1-46dd-a979-f38a4372a573 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:17.594253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690d02e-36c1-46dd-a979-f38a4372a573 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:17.595608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:17.601062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab7b95-6666-402b-81f8-8a0db1a12e05 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:17.601498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab7b95-6666-402b-81f8-8a0db1a12e05 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:17.601598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab7b95-6666-402b-81f8-8a0db1a12e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:17.649255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:17.667026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ccab7b95-6666-402b-81f8-8a0db1a12e05 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:17.668284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ccab7b95-6666-402b-81f8-8a0db1a12e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:17.763915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:18.051604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690d02e-36c1-46dd-a979-f38a4372a573 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:18.053158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0690d02e-36c1-46dd-a979-f38a4372a573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:18.156791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690d02e-36c1-46dd-a979-f38a4372a573 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:18.197958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:18.232532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:18.233132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:18.233406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:18.233674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:18.235298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:18.345390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:18.346755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e076b-b52e-4c3e-bd83-e9a56c24069b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:18.364847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 736/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:20.673351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c3c57-0305-4ee9-b218-0103299e4f64 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:20.673891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c3c57-0305-4ee9-b218-0103299e4f64 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:20.673891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c3c57-0305-4ee9-b218-0103299e4f64 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:20.675277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:20.681207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74d4db-a840-4f0f-8666-4aa898d7a3d8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:20.681761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74d4db-a840-4f0f-8666-4aa898d7a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:20.681761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74d4db-a840-4f0f-8666-4aa898d7a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:20.751101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4e74d4db-a840-4f0f-8666-4aa898d7a3d8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:20.752382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4e74d4db-a840-4f0f-8666-4aa898d7a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:20.774070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:20.850844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:21.145675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c3c57-0305-4ee9-b218-0103299e4f64 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:21.147306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b99c3c57-0305-4ee9-b218-0103299e4f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:21.259350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c3c57-0305-4ee9-b218-0103299e4f64 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:21.280443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:21.334921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:21.335564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:21.335846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:21.336119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:21.337724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:21.455742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:21.457134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffb298-6e8f-4713-812b-8b44ef0dfece service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:21.475289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 715/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:23.705842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724b7d-bc68-4269-ac97-9ff587b89e8a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:23.706414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724b7d-bc68-4269-ac97-9ff587b89e8a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:23.706514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724b7d-bc68-4269-ac97-9ff587b89e8a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:23.707912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:23.713770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa2d2-5b42-4347-a3e7-09a36f4b57b0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:23.714190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa2d2-5b42-4347-a3e7-09a36f4b57b0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:23.714470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa2d2-5b42-4347-a3e7-09a36f4b57b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:23.780317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:23.783896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b22fa2d2-5b42-4347-a3e7-09a36f4b57b0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:23.785785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b22fa2d2-5b42-4347-a3e7-09a36f4b57b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:23.882348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:24.182769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724b7d-bc68-4269-ac97-9ff587b89e8a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:24.184474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e7724b7d-bc68-4269-ac97-9ff587b89e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:24.295606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724b7d-bc68-4269-ac97-9ff587b89e8a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:24.345855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:24.370374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:24.370975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:24.371282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:24.371560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:24.373147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:24.480468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:24.481989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820b10d-2086-463e-84bb-d4d6dbb8042c service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:24.501127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 739/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:26.722407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697cd9-a6de-43ee-b481-542dba0fa7d7 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:26.722975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697cd9-a6de-43ee-b481-542dba0fa7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:26.722975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61697cd9-a6de-43ee-b481-542dba0fa7d7 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:26.724390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:26.729969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979b59-dcba-4b63-9aaa-b67792b1c094 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:26.730379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979b59-dcba-4b63-9aaa-b67792b1c094 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:26.730487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979b59-dcba-4b63-9aaa-b67792b1c094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:26.797503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e0979b59-dcba-4b63-9aaa-b67792b1c094 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:26.798732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e0979b59-dcba-4b63-9aaa-b67792b1c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:26.826965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:26.895506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:27.186680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697cd9-a6de-43ee-b481-542dba0fa7d7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:27.188756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-61697cd9-a6de-43ee-b481-542dba0fa7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:27.292463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61697cd9-a6de-43ee-b481-542dba0fa7d7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:27.320769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:27.365464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:27.365979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:27.366251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:27.366516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:27.368127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:27.475701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:27.477130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdade3-d8db-4bf2-919e-75e634a8a59a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:27.495481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 718/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:29.729008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c8cb-69a2-4754-a949-70fb9a56744e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:29.729550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c8cb-69a2-4754-a949-70fb9a56744e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:29.729550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c8cb-69a2-4754-a949-70fb9a56744e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:29.730847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:29.736283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576a76-deb1-4e8d-88f8-830664579524 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:29.736733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576a76-deb1-4e8d-88f8-830664579524 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:29.736844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576a76-deb1-4e8d-88f8-830664579524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:29.804173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ac576a76-deb1-4e8d-88f8-830664579524 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:29.805399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ac576a76-deb1-4e8d-88f8-830664579524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:29.813560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:29.904140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:30.192988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c8cb-69a2-4754-a949-70fb9a56744e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:30.194576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3921c8cb-69a2-4754-a949-70fb9a56744e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:30.299673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c8cb-69a2-4754-a949-70fb9a56744e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:30.356246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:30.380889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:30.381495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:30.381780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:30.382052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:30.383676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:30.491945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:30.493424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926ad13-5a55-41e0-ad8b-7612f523813e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:30.512321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 742/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:32.715968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf019c-55ff-4eee-948a-b951c893e36b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:32.716547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf019c-55ff-4eee-948a-b951c893e36b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:32.716547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf019c-55ff-4eee-948a-b951c893e36b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:32.717937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:32.723928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eba1a-44d2-4b19-9956-811b20a25f9e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:32.724378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eba1a-44d2-4b19-9956-811b20a25f9e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:32.724474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9eba1a-44d2-4b19-9956-811b20a25f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:32.791866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9f9eba1a-44d2-4b19-9956-811b20a25f9e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:32.793112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f9eba1a-44d2-4b19-9956-811b20a25f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:32.832120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:32.890530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:33.180150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf019c-55ff-4eee-948a-b951c893e36b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:33.181674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cedf019c-55ff-4eee-948a-b951c893e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:33.285282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf019c-55ff-4eee-948a-b951c893e36b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:33.322521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:33.358433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:33.359029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:33.359335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:33.360111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:33.361698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:33.469860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:33.471251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eea6c-dae7-426f-90e6-b0ca5eb7d888 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:33.490165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 721/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:35.701991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1e5a1-f166-4a12-85ae-80dd66988997 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:35.702554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1e5a1-f166-4a12-85ae-80dd66988997 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:35.702554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1e5a1-f166-4a12-85ae-80dd66988997 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:35.703960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:35.709375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ccd5-59e4-46b6-af1f-5f0b13d3fa41 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:35.709796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ccd5-59e4-46b6-af1f-5f0b13d3fa41 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:35.709989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ccd5-59e4-46b6-af1f-5f0b13d3fa41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:35.777658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fa41ccd5-59e4-46b6-af1f-5f0b13d3fa41 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:35.778781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa41ccd5-59e4-46b6-af1f-5f0b13d3fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:35.796682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:35.874228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:36.163562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1e5a1-f166-4a12-85ae-80dd66988997 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:36.165098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6dd1e5a1-f166-4a12-85ae-80dd66988997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:36.273096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1e5a1-f166-4a12-85ae-80dd66988997 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:36.340042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:36.347659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:36.348270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:36.348546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:36.348815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:36.350413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:36.457250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:36.458616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98549074-7133-4ba2-9402-4c7926db437e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:36.477300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 745/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:38.676543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481584d-ac86-44f1-bc20-49703aae97f6 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:38.677109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481584d-ac86-44f1-bc20-49703aae97f6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:38.677109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481584d-ac86-44f1-bc20-49703aae97f6 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:38.678436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:38.683849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eba7f-21ea-4a31-846a-cdae3c9583a5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:38.684259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eba7f-21ea-4a31-846a-cdae3c9583a5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:38.684437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eba7f-21ea-4a31-846a-cdae3c9583a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:38.765815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ea8eba7f-21ea-4a31-846a-cdae3c9583a5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:38.766932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ea8eba7f-21ea-4a31-846a-cdae3c9583a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:38.816272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:38.864619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:52:39.156958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481584d-ac86-44f1-bc20-49703aae97f6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:39.158504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1481584d-ac86-44f1-bc20-49703aae97f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:39.262746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481584d-ac86-44f1-bc20-49703aae97f6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:39.310775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:39.336945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:39.337555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:39.337838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:39.338106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:39.339732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:39.453806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:39.455162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387a07b-5838-483a-ab62-f3e9dc2a6253 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:39.473448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 724/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:48.863053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=22 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:49.471255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=66 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:59.484140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf4aa-ae2c-493e-a3fa-d9dfdd7c3be0 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:59.484701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf4aa-ae2c-493e-a3fa-d9dfdd7c3be0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:59.484701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cf4aa-ae2c-493e-a3fa-d9dfdd7c3be0 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:59.486016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 747/1471] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:52:59.491602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824f889-3b23-4d01-8cc1-90148562d1d0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:59.492270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824f889-3b23-4d01-8cc1-90148562d1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:59.492270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824f889-3b23-4d01-8cc1-90148562d1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:59.558420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6824f889-3b23-4d01-8cc1-90148562d1d0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:59.559675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6824f889-3b23-4d01-8cc1-90148562d1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:59.665242 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 725/1472] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:59.972569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c2035-e734-4ddb-912a-66b6d4bdeb94 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:59.972855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c2035-e734-4ddb-912a-66b6d4bdeb94 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:59.973142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c2035-e734-4ddb-912a-66b6d4bdeb94 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:59.974525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 748/1473] 10.222.0.66 () {54 vars in 962 bytes} [Thu Sep 22 14:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:59.980235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf216-9cdf-4520-9fb0-7b19b87f993b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:59.980681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf216-9cdf-4520-9fb0-7b19b87f993b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:59.981000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf216-9cdf-4520-9fb0-7b19b87f993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:00.088385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-34bbf216-9cdf-4520-9fb0-7b19b87f993b None None] MFA Rules not processed for user `623f9046c3924b35b36d8807f8eef54c`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:00.089550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-34bbf216-9cdf-4520-9fb0-7b19b87f993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:00.112767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:00.174047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 726/1474] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Sep 22 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:53:00.182167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c2035-e734-4ddb-912a-66b6d4bdeb94 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:00.183782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ef6c2035-e734-4ddb-912a-66b6d4bdeb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:00.309901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c2035-e734-4ddb-912a-66b6d4bdeb94 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:00.406685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:00.407710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:00.408218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:00.408706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:00.411880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:00.488805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:00.551015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:00.552509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a0f99-d1b7-431c-98c0-e3d2b460b7ac service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:00.573302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 749/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:02.918630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ef02f-ad4d-47be-9003-6da45b9fbebf None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:02.919248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ef02f-ad4d-47be-9003-6da45b9fbebf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:02.919248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ef02f-ad4d-47be-9003-6da45b9fbebf None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:02.920564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 727/1476] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:02.926715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67822ecb-b784-43fd-b7c4-786c572abf7a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:02.927191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67822ecb-b784-43fd-b7c4-786c572abf7a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:02.927307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67822ecb-b784-43fd-b7c4-786c572abf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:03.000124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-67822ecb-b784-43fd-b7c4-786c572abf7a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:03.001359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-67822ecb-b784-43fd-b7c4-786c572abf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:03.115002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 750/1477] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:53:03.390937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ef02f-ad4d-47be-9003-6da45b9fbebf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:03.392539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c58ef02f-ad4d-47be-9003-6da45b9fbebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:03.529049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ef02f-ad4d-47be-9003-6da45b9fbebf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:03.583443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:03.638612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:03.639246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:03.639531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:03.639803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:03.640505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:03.642351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:53:03.642351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:53:03.642351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:53:03.643464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aecca1-563f-431b-a232-45556e08ae13 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:03.663774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 728/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Sep 22 14:53:03 2022] GET /identity/v3/projects => generated 2553 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:53:05.932074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdee46f-71f3-4df6-b812-83a7015da0ab None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:05.932756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdee46f-71f3-4df6-b812-83a7015da0ab None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:05.932972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdee46f-71f3-4df6-b812-83a7015da0ab None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:05.935338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 751/1479] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:53:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:05.942761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ab12-20ab-4728-8ba3-fd448d79af39 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:05.943314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ab12-20ab-4728-8ba3-fd448d79af39 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:05.943679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ab12-20ab-4728-8ba3-fd448d79af39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:06.031653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f012ab12-20ab-4728-8ba3-fd448d79af39 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:06.032897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f012ab12-20ab-4728-8ba3-fd448d79af39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:06.147823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 729/1480] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:06.448205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdee46f-71f3-4df6-b812-83a7015da0ab None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:06.449770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dfdee46f-71f3-4df6-b812-83a7015da0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:06.459954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:06.619859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdee46f-71f3-4df6-b812-83a7015da0ab None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:06.711969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:06.750606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:06.751247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:06.751540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:06.751818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:06.753530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:06.860561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:06.862236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9bfa8-66aa-4741-886e-d02cc30b7a0c service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:06.881529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 752/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:09.343180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b91857-c8bc-4166-9a70-f8ae1102870a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:09.343786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b91857-c8bc-4166-9a70-f8ae1102870a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:09.343786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b91857-c8bc-4166-9a70-f8ae1102870a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:09.345183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 730/1482] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:09.350799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e50a5-6ade-4551-8899-6aab0ef677ce None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:09.351261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e50a5-6ade-4551-8899-6aab0ef677ce None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:09.351400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e50a5-6ade-4551-8899-6aab0ef677ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:09.417416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3f2e50a5-6ade-4551-8899-6aab0ef677ce None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:09.418629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3f2e50a5-6ade-4551-8899-6aab0ef677ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:09.466256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:09.518071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 753/1483] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:09.810613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b91857-c8bc-4166-9a70-f8ae1102870a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:09.812720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b3b91857-c8bc-4166-9a70-f8ae1102870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:09.838094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:09.922619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b91857-c8bc-4166-9a70-f8ae1102870a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:10.012947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:10.013556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:10.013832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:10.014112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:10.017204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:10.111015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:10.153276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:10.154728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3e7b8-c2b6-40ed-a1bb-d10893bd9319 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:10.174326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 731/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:12.502993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbeb6d-46cf-46eb-8561-f5b8b99f1280 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:12.503663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbeb6d-46cf-46eb-8561-f5b8b99f1280 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:12.503663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbeb6d-46cf-46eb-8561-f5b8b99f1280 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:12.505020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 754/1485] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:12.512156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9a48-03fe-4304-9b33-8d57741e80c3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:12.512739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9a48-03fe-4304-9b33-8d57741e80c3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:12.512899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc9a48-03fe-4304-9b33-8d57741e80c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:12.604173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-77fc9a48-03fe-4304-9b33-8d57741e80c3 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:12.605477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-77fc9a48-03fe-4304-9b33-8d57741e80c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:12.722358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 732/1486] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:13.032146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbeb6d-46cf-46eb-8561-f5b8b99f1280 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:13.033712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-acfbeb6d-46cf-46eb-8561-f5b8b99f1280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:13.156715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbeb6d-46cf-46eb-8561-f5b8b99f1280 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:13.238523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:13.240172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:13.240928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:13.241245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:13.241685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:13.243901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:13.360825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:13.362307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8dedf-1f5d-4d01-bb8f-4cabda7b5fb5 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:13.381391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 755/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:15.709138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0558f-c352-466d-afd0-7be763f9f36d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:15.709719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0558f-c352-466d-afd0-7be763f9f36d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:15.709719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0558f-c352-466d-afd0-7be763f9f36d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:15.711025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 733/1488] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:15.716986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc8046-f022-40b6-a6bb-578ed3f4a971 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:15.717470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc8046-f022-40b6-a6bb-578ed3f4a971 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:15.717581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc8046-f022-40b6-a6bb-578ed3f4a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:15.786736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ebcc8046-f022-40b6-a6bb-578ed3f4a971 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:15.787918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ebcc8046-f022-40b6-a6bb-578ed3f4a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:15.842574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:15.895904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 756/1489] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:16.187406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0558f-c352-466d-afd0-7be763f9f36d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:16.188953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e6d0558f-c352-466d-afd0-7be763f9f36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:16.230820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:16.301785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0558f-c352-466d-afd0-7be763f9f36d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:16.375899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:16.376626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:16.376855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:16.377114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:16.378701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:16.465355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:16.498372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:16.500371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9b4af-f893-49e7-8520-f434f81e3957 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:16.520932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 734/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:19.008307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b33329-944a-42d8-aad9-0eea40a5c553 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:19.008925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b33329-944a-42d8-aad9-0eea40a5c553 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:19.008925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b33329-944a-42d8-aad9-0eea40a5c553 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:19.010465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 757/1491] 10.222.0.66 () {54 vars in 927 bytes} [Thu Sep 22 14:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:19.016786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ef890-bd17-4446-8245-a06f18997064 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:19.017287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ef890-bd17-4446-8245-a06f18997064 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:19.017395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ef890-bd17-4446-8245-a06f18997064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:19.088140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-079ef890-bd17-4446-8245-a06f18997064 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:19.089372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-079ef890-bd17-4446-8245-a06f18997064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:19.218749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 735/1492] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:19.515021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b33329-944a-42d8-aad9-0eea40a5c553 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:19.516648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96b33329-944a-42d8-aad9-0eea40a5c553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:19.740189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b33329-944a-42d8-aad9-0eea40a5c553 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:19.814182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:19.814782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:19.815058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:19.815357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:19.816965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:19.826541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:19.933574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:19.935140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a2786-b539-4371-a9f1-8df5af7c3a37 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:19.954596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 758/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:53:29.216905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=82 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:29.952693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=50 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:34.734847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:34.735433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:34.735433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:34.809886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:34.811084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:34.912922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 736/1494] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:34.921038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:34.921524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:34.921655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:34.996063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:34.997211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.093898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 759/1495] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:35.098550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.099805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.111122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fba05-9205-4bf3-a454-384b0a310710 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.172919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.208081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.208688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.208966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.209232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.209866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:54:35.210166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.211172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.211172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:54:35.211172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:54:35.212094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.223127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 737/1496] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:54:35.231411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.240368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b5614-bad7-4a92-a22b-8a612216daf9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.301677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.335316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.335916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.336202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.336469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.337098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.338824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.338824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:54:35.338824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:54:35.340059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.396815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 760/1497] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:54:35 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:54:35.400841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.402361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.434696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec97e-3657-45f3-9308-cd50ff495f16 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.531029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.531664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.531944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.532219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.532839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.533851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.533851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:54:35.533851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:54:35.535066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.579139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 738/1498] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:54:35 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:54:35.583724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.585216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.645578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021abcf-6191-46f7-952a-22c6ebf525a6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.728894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.740559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.741175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.741455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.741727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.742366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.743447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.743447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:54:35.743447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:54:35.744340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.757525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 761/1499] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:54:35.761953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.764071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.775418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14ddec-ce64-47d5-a16a-06eb9a8a0b45 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.800358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.869794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.870388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.870670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.870987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.871606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.872669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.872669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:54:35.872669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:54:35.873591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038ca79-48a7-4e1d-8224-0e06cf2b686e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.886260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 739/1500] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:54:35.891575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.893645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.931400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d93d3-f2f0-4a5a-b4c2-e28cf11f40d7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:36.025295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:36.025947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.026214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.026478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] PATH_INFO: `/v3/projects/0aaafda21a2d47dfa8dc420dad06ebb0/users/c80eb82011904206b18f50c24dc0476b/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.048525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aaafda21a2d47dfa8dc420dad06ebb0, user_id=c80eb82011904206b18f50c24dc0476b, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:36.049540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:36.049540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:54:36.049540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:54:36.051502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e275a-8b9d-4233-957b-c79e287be1f5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:36.078693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 762/1501] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:54:35 2022] PUT /identity/v3/projects/0aaafda21a2d47dfa8dc420dad06ebb0/users/c80eb82011904206b18f50c24dc0476b/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:54:36.090791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67bca1-3bbd-4050-b6d2-b9251dc3bd46 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.091101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67bca1-3bbd-4050-b6d2-b9251dc3bd46 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.091456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67bca1-3bbd-4050-b6d2-b9251dc3bd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.149824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ae67bca1-3bbd-4050-b6d2-b9251dc3bd46 None None] MFA Rules not processed for user `c80eb82011904206b18f50c24dc0476b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:36.150962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ae67bca1-3bbd-4050-b6d2-b9251dc3bd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.227167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 740/1502] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:36.240591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013643c-49ff-4aa4-b337-3289d9166c4b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.240881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013643c-49ff-4aa4-b337-3289d9166c4b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.241158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013643c-49ff-4aa4-b337-3289d9166c4b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.242505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 763/1503] 10.222.0.66 () {54 vars in 963 bytes} [Thu Sep 22 14:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:54:36.247501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f24d13-c3e5-43cd-a340-51effe4a4dff None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.248348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f24d13-c3e5-43cd-a340-51effe4a4dff None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.248626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f24d13-c3e5-43cd-a340-51effe4a4dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.281985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.327746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f3f24d13-c3e5-43cd-a340-51effe4a4dff None None] MFA Rules not processed for user `b4e09c957a104f2299ab55d8283124c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:36.328958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3f24d13-c3e5-43cd-a340-51effe4a4dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.403462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 741/1504] 10.222.0.66 () {58 vars in 1060 bytes} [Thu Sep 22 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:36.412863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013643c-49ff-4aa4-b337-3289d9166c4b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:36.414372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7013643c-49ff-4aa4-b337-3289d9166c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.463936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.519271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013643c-49ff-4aa4-b337-3289d9166c4b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:36.591562 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:36.592164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.592443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.592713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.594302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.697922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:36.699296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:36.702757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.718264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 764/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:54:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:54:36.765529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74443318-8d37-4c84-9776-dca186fcef92 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.766107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74443318-8d37-4c84-9776-dca186fcef92 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.766107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74443318-8d37-4c84-9776-dca186fcef92 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.767399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 742/1506] 10.222.0.66 () {52 vars in 830 bytes} [Thu Sep 22 14:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:54:36.771226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:36.772439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.874445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fefe0-f8ab-4b1d-8f3c-88743b9c200c service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:36.968779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:36.969539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.969647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.969922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] PATH_INFO: `/v3/users/c80eb82011904206b18f50c24dc0476b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.986152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80eb82011904206b18f50c24dc0476b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:36.988055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:36.988055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:54:36.988055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:54:36.990349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:37.000800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:37.061069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80eb82011904206b18f50c24dc0476b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.070729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80eb82011904206b18f50c24dc0476b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.096208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80eb82011904206b18f50c24dc0476b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.096789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80eb82011904206b18f50c24dc0476b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.097211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80eb82011904206b18f50c24dc0476b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.115817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 765/1507] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:54:36 2022] DELETE /identity/v3/users/c80eb82011904206b18f50c24dc0476b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:54:37.120371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:37.120652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:37.120932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:37.179270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:37.180478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.254990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:37.269222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 743/1508] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:37.278783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:37.280724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.377907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee699ba-7d68-44c7-adf4-af91e366be02 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:37.432166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:37.452301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:37.452905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:37.453183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:37.453451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:37.455007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.564713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:37.566255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb9eb5-3824-4f85-a085-0db95c080754 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:37.585025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 766/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:54:37.937846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:37.939112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.950442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fcfeb-ba45-4186-897c-3dcce04c07a1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:38.049546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:38.050179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:38.050453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:38.050737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] PATH_INFO: `/v3/projects/0aaafda21a2d47dfa8dc420dad06ebb0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:38.061266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aaafda21a2d47dfa8dc420dad06ebb0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:38.062286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:38.062286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:54:38.062286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:54:38.063537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:38.131872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aaafda21a2d47dfa8dc420dad06ebb0', 'request_id': 'req-15ce7320-b986-4dcc-82fe-5d2a07640e86'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.142089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aaafda21a2d47dfa8dc420dad06ebb0', 'request_id': 'req-15ce7320-b986-4dcc-82fe-5d2a07640e86'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.148170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aaafda21a2d47dfa8dc420dad06ebb0', 'request_id': 'req-15ce7320-b986-4dcc-82fe-5d2a07640e86'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.179079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] The token cache is being invalidate because project 0aaafda21a2d47dfa8dc420dad06ebb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:54:38.179397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.179826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15ce7320-b986-4dcc-82fe-5d2a07640e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.181229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 744/1510] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:54:37 2022] DELETE /identity/v3/projects/0aaafda21a2d47dfa8dc420dad06ebb0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:54:47.582749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=62 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:48.177008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:10.268368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bc64a-d49c-498f-b7a0-5c19d1fc37eb None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:10.268939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bc64a-d49c-498f-b7a0-5c19d1fc37eb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:10.268939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bc64a-d49c-498f-b7a0-5c19d1fc37eb None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:10.270193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:10.275251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ac19b-74bb-48f5-87e9-debf0dd135b7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:10.275698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ac19b-74bb-48f5-87e9-debf0dd135b7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:10.275848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ac19b-74bb-48f5-87e9-debf0dd135b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:10.356554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a00ac19b-74bb-48f5-87e9-debf0dd135b7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:10.357774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a00ac19b-74bb-48f5-87e9-debf0dd135b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:10.463325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:10.618982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bc64a-d49c-498f-b7a0-5c19d1fc37eb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:10.621143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5a2bc64a-d49c-498f-b7a0-5c19d1fc37eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:10.742632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bc64a-d49c-498f-b7a0-5c19d1fc37eb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:10.748169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3b3b-44f7-4e6e-b8f1-b43f08ee54be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:10.748779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3b3b-44f7-4e6e-b8f1-b43f08ee54be service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:10.749060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3b3b-44f7-4e6e-b8f1-b43f08ee54be service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:10.749335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3b3b-44f7-4e6e-b8f1-b43f08ee54be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:10.757366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3b3b-44f7-4e6e-b8f1-b43f08ee54be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:10.758949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3b3b-44f7-4e6e-b8f1-b43f08ee54be service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:10.768616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 768/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:11.854646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a8c4e-97dd-4922-ba9e-0e215a9e8c0a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:11.855231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a8c4e-97dd-4922-ba9e-0e215a9e8c0a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:11.855231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a8c4e-97dd-4922-ba9e-0e215a9e8c0a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:11.856575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:11.863530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0822a-63f4-46f0-aed3-8e831163a2e2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:11.864006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0822a-63f4-46f0-aed3-8e831163a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:11.864108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0822a-63f4-46f0-aed3-8e831163a2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:11.883919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1ee0822a-63f4-46f0-aed3-8e831163a2e2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:11.885148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1ee0822a-63f4-46f0-aed3-8e831163a2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:11.893358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:56:12.053526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a8c4e-97dd-4922-ba9e-0e215a9e8c0a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:12.068586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a8c4e-97dd-4922-ba9e-0e215a9e8c0a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:12.074320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131075e4-d147-4c87-ab24-ecf19e15262a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:12.074912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131075e4-d147-4c87-ab24-ecf19e15262a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:12.075194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131075e4-d147-4c87-ab24-ecf19e15262a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:12.075490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131075e4-d147-4c87-ab24-ecf19e15262a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:12.083732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131075e4-d147-4c87-ab24-ecf19e15262a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:12.085316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131075e4-d147-4c87-ab24-ecf19e15262a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:12.094678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 747/1516] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:12.841337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0ea0-b264-4603-a94f-07e9cd5e2af7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:12.841606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0ea0-b264-4603-a94f-07e9cd5e2af7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:12.841883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0ea0-b264-4603-a94f-07e9cd5e2af7 None None] PATH_INFO: `/v3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:12.843167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 770/1517] 10.222.0.49 () {54 vars in 925 bytes} [Thu Sep 22 14:56:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:56:12.851125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee49cb-1ebe-403e-a894-dbc5f20f14bb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:12.851458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee49cb-1ebe-403e-a894-dbc5f20f14bb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:12.851727 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee49cb-1ebe-403e-a894-dbc5f20f14bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:12.913084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7eee49cb-1ebe-403e-a894-dbc5f20f14bb None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:12.914300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7eee49cb-1ebe-403e-a894-dbc5f20f14bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:12.934728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:12.994013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 748/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Thu Sep 22 14:56:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:21.878969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=56 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:22.601988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46b7a-0546-493a-8eee-91a76a860e7e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:22.602270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46b7a-0546-493a-8eee-91a76a860e7e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:22.602540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46b7a-0546-493a-8eee-91a76a860e7e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:22.603953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 771/1519] 10.222.0.66 () {54 vars in 961 bytes} [Thu Sep 22 14:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:22.608887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65585-a573-4494-ae21-9d8ccbb0ef08 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:22.609371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65585-a573-4494-ae21-9d8ccbb0ef08 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:22.610395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65585-a573-4494-ae21-9d8ccbb0ef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:22.630125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b6c65585-a573-4494-ae21-9d8ccbb0ef08 None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:22.631350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b6c65585-a573-4494-ae21-9d8ccbb0ef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:22.644274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 749/1520] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Sep 22 14:56:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:56:22.651949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46b7a-0546-493a-8eee-91a76a860e7e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:22.665464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46b7a-0546-493a-8eee-91a76a860e7e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:22.670416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739f153-0658-4c85-891b-7ad86e173edc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:22.671040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739f153-0658-4c85-891b-7ad86e173edc service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:22.671354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739f153-0658-4c85-891b-7ad86e173edc service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:22.671624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739f153-0658-4c85-891b-7ad86e173edc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:22.680737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739f153-0658-4c85-891b-7ad86e173edc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:22.682334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739f153-0658-4c85-891b-7ad86e173edc service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:22.692760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 772/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:23.600577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed46b-24e4-45b0-a783-a66adf35a4e5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:23.600851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed46b-24e4-45b0-a783-a66adf35a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:23.601118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eed46b-24e4-45b0-a783-a66adf35a4e5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:23.602511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:23.608609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528f968-145d-4cd4-ad72-8851f2f7acce None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:23.609080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528f968-145d-4cd4-ad72-8851f2f7acce None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:23.609216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528f968-145d-4cd4-ad72-8851f2f7acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:23.629131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4528f968-145d-4cd4-ad72-8851f2f7acce None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:23.630315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4528f968-145d-4cd4-ad72-8851f2f7acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:23.638735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:56:23.794827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed46b-24e4-45b0-a783-a66adf35a4e5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:23.810821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eed46b-24e4-45b0-a783-a66adf35a4e5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:23.816918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6858b0-2a71-472a-9943-9f7742284c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:23.817509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6858b0-2a71-472a-9943-9f7742284c6e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:23.817786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6858b0-2a71-472a-9943-9f7742284c6e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:23.818055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6858b0-2a71-472a-9943-9f7742284c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:23.826198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6858b0-2a71-472a-9943-9f7742284c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:23.827797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6858b0-2a71-472a-9943-9f7742284c6e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:23.836882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 751/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:24.920036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8a967-ae44-4b40-b5fc-cce55a1e2af9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:24.920755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8a967-ae44-4b40-b5fc-cce55a1e2af9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:24.920755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8a967-ae44-4b40-b5fc-cce55a1e2af9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:24.921998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:24.928526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533e3e6-430e-4a66-8e84-5c35bca3b6a4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:24.929008 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533e3e6-430e-4a66-8e84-5c35bca3b6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:24.929114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533e3e6-430e-4a66-8e84-5c35bca3b6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:24.948933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d533e3e6-430e-4a66-8e84-5c35bca3b6a4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:24.950137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d533e3e6-430e-4a66-8e84-5c35bca3b6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:24.958625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:25.109149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8a967-ae44-4b40-b5fc-cce55a1e2af9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:25.123302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8a967-ae44-4b40-b5fc-cce55a1e2af9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:25.132428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca66f31-ff18-4a43-8164-c2e47ef70010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:25.133033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66f31-ff18-4a43-8164-c2e47ef70010 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:25.133318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66f31-ff18-4a43-8164-c2e47ef70010 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:25.133590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66f31-ff18-4a43-8164-c2e47ef70010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:25.143273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca66f31-ff18-4a43-8164-c2e47ef70010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:25.144871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca66f31-ff18-4a43-8164-c2e47ef70010 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:25.154280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 775/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:29.198721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9744ae-f1aa-49b7-9ba6-caab8499511b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:29.199327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9744ae-f1aa-49b7-9ba6-caab8499511b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:29.199327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9744ae-f1aa-49b7-9ba6-caab8499511b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:29.200655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:29.206404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af20a38-7f73-401e-b38a-349a13d5fddc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:29.206872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af20a38-7f73-401e-b38a-349a13d5fddc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:29.206978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af20a38-7f73-401e-b38a-349a13d5fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:29.228072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0af20a38-7f73-401e-b38a-349a13d5fddc None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:29.229282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0af20a38-7f73-401e-b38a-349a13d5fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:29.237245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:29.521213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9744ae-f1aa-49b7-9ba6-caab8499511b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:29.522989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1d9744ae-f1aa-49b7-9ba6-caab8499511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:29.605766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9744ae-f1aa-49b7-9ba6-caab8499511b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:29.609393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802dde8-bf8d-4320-96cb-427764fd8855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:29.610040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802dde8-bf8d-4320-96cb-427764fd8855 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:29.610320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802dde8-bf8d-4320-96cb-427764fd8855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:29.610595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802dde8-bf8d-4320-96cb-427764fd8855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:29.619789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802dde8-bf8d-4320-96cb-427764fd8855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:29.621231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802dde8-bf8d-4320-96cb-427764fd8855 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:29.631228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 754/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:31.805099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3d82e-50fd-4cce-baf7-10e569ad29a4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:31.805717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3d82e-50fd-4cce-baf7-10e569ad29a4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:31.805717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3d82e-50fd-4cce-baf7-10e569ad29a4 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:31.807113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:31.812637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0907a-ce95-419e-be5e-57d5057fa033 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:31.813052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0907a-ce95-419e-be5e-57d5057fa033 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:31.813190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0907a-ce95-419e-be5e-57d5057fa033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:31.832459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3fb0907a-ce95-419e-be5e-57d5057fa033 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:31.833672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3fb0907a-ce95-419e-be5e-57d5057fa033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:31.841493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:32.135146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3d82e-50fd-4cce-baf7-10e569ad29a4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:32.146560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3d82e-50fd-4cce-baf7-10e569ad29a4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:32.150101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57498874-d6ac-4723-a142-7ff4f29baa50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:32.150694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57498874-d6ac-4723-a142-7ff4f29baa50 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:32.150975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57498874-d6ac-4723-a142-7ff4f29baa50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:32.151281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57498874-d6ac-4723-a142-7ff4f29baa50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:32.161764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57498874-d6ac-4723-a142-7ff4f29baa50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:32.163233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57498874-d6ac-4723-a142-7ff4f29baa50 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:32.173247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 778/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:35.989310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058962a6-0226-4c24-8b5b-c19245afbf74 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:35.989863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058962a6-0226-4c24-8b5b-c19245afbf74 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:35.989863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058962a6-0226-4c24-8b5b-c19245afbf74 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:35.991219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:35.996363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebc69d-2d7f-4e47-afcd-412e253a9df4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:35.996792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebc69d-2d7f-4e47-afcd-412e253a9df4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:35.996917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebc69d-2d7f-4e47-afcd-412e253a9df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:36.016794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c5ebc69d-2d7f-4e47-afcd-412e253a9df4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:36.017989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c5ebc69d-2d7f-4e47-afcd-412e253a9df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:36.025870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:36.328421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058962a6-0226-4c24-8b5b-c19245afbf74 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:36.330128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-058962a6-0226-4c24-8b5b-c19245afbf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:36.394371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:36.408834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058962a6-0226-4c24-8b5b-c19245afbf74 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:36.412420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdced2c-91dd-4cd8-aaa0-84548999a824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:36.413018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdced2c-91dd-4cd8-aaa0-84548999a824 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:36.413297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdced2c-91dd-4cd8-aaa0-84548999a824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:36.413569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdced2c-91dd-4cd8-aaa0-84548999a824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:36.423015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdced2c-91dd-4cd8-aaa0-84548999a824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:36.424488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdced2c-91dd-4cd8-aaa0-84548999a824 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:36.433841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 757/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:38.572166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713444c3-b311-44d1-a97b-e2010bbaead6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:38.572680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713444c3-b311-44d1-a97b-e2010bbaead6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:38.572680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713444c3-b311-44d1-a97b-e2010bbaead6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:38.573968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:38.579855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef5142-7b72-4e07-a8e1-82c3895ee7e1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:38.580270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef5142-7b72-4e07-a8e1-82c3895ee7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:38.580375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef5142-7b72-4e07-a8e1-82c3895ee7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:38.599488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-00ef5142-7b72-4e07-a8e1-82c3895ee7e1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:38.600670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-00ef5142-7b72-4e07-a8e1-82c3895ee7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:38.608476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:38.907528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713444c3-b311-44d1-a97b-e2010bbaead6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:38.909279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-713444c3-b311-44d1-a97b-e2010bbaead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:38.922778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713444c3-b311-44d1-a97b-e2010bbaead6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:38.926368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0e9ea-d749-4f79-bdc3-4cfbca83b529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:38.926959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0e9ea-d749-4f79-bdc3-4cfbca83b529 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:38.927266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0e9ea-d749-4f79-bdc3-4cfbca83b529 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:38.927542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0e9ea-d749-4f79-bdc3-4cfbca83b529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:38.936543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0e9ea-d749-4f79-bdc3-4cfbca83b529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:38.939162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0e9ea-d749-4f79-bdc3-4cfbca83b529 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:38.953916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 781/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:56:48.594719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=11 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:48.948363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=33 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:06.001558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.002237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.002237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.034506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.035781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.050836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 759/1540] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:06.060102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a036e39-208b-416e-a7b8-5ac2d78ec1c1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.061808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a036e39-208b-416e-a7b8-5ac2d78ec1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.061808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a036e39-208b-416e-a7b8-5ac2d78ec1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.092431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6a036e39-208b-416e-a7b8-5ac2d78ec1c1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.094559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6a036e39-208b-416e-a7b8-5ac2d78ec1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.104718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 782/1541] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.109670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.121739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbd634-6509-402d-a70c-35c64a9d36ff None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.136195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.136842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.137124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.137399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.138966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:06.139441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.142092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.156338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 760/1542] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:06.165534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8b215-c19e-4ca3-bf52-8262f2f92f63 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.165778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.165890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8b215-c19e-4ca3-bf52-8262f2f92f63 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.166257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8b215-c19e-4ca3-bf52-8262f2f92f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.177403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6a0c7-fb51-462a-a240-cd80b65f04a8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.199313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.199957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.200275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.200550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.201275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.203192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:06.203192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:58:06.203192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:58:06.204957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.205899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fdb8b215-c19e-4ca3-bf52-8262f2f92f63 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.207340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fdb8b215-c19e-4ca3-bf52-8262f2f92f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.226861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 783/1543] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.236864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a36e9-68c0-42bd-8efa-bcaf06632b8f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.237289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a36e9-68c0-42bd-8efa-bcaf06632b8f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.237577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a36e9-68c0-42bd-8efa-bcaf06632b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.279765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-746a36e9-68c0-42bd-8efa-bcaf06632b8f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.280031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 761/1544] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:06.281227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-746a36e9-68c0-42bd-8efa-bcaf06632b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.284118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.384229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 784/1545] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:06.385796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.386085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.386361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.417653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3c72-8efc-4525-8398-7093500bb3bd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.423936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.425553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.433499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183c1d1-ce0d-4338-8002-defa50a7fea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.434127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183c1d1-ce0d-4338-8002-defa50a7fea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.434562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183c1d1-ce0d-4338-8002-defa50a7fea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.434847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183c1d1-ce0d-4338-8002-defa50a7fea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.435519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183c1d1-ce0d-4338-8002-defa50a7fea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.437507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183c1d1-ce0d-4338-8002-defa50a7fea9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.441274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 785/1546] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.442191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.454851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f91a3-ce36-48c6-aaa2-198e77e5639d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.468943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.469606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.469921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.470202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.470903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:06.471224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.472567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:06.472567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:58:06.472567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:58:06.473465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f974c-3237-4120-aa84-1d3f319a41ca admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.487810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 762/1547] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:06.489099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d125571-9eef-4255-8511-f39bc0f2c237 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.489397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d125571-9eef-4255-8511-f39bc0f2c237 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.489678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d125571-9eef-4255-8511-f39bc0f2c237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.498369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 786/1548] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:06.499750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.500024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.500300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.529773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.531622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.550842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 787/1549] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.551734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.552579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4d125571-9eef-4255-8511-f39bc0f2c237 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.554027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4d125571-9eef-4255-8511-f39bc0f2c237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.568609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2b14c-07d1-459c-a3ac-30432bf08d37 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.571561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 763/1550] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.572911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8e813-fd7c-45df-baa9-18aa1cdc30f2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.573214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8e813-fd7c-45df-baa9-18aa1cdc30f2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.573582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8e813-fd7c-45df-baa9-18aa1cdc30f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.573806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.574354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.574643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.574897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.575586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.577190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.588042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 788/1551] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:06.589012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.593995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1ee8e813-fd7c-45df-baa9-18aa1cdc30f2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.595349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1ee8e813-fd7c-45df-baa9-18aa1cdc30f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.598850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d8e3f-bf18-4dcd-88c0-910738726631 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.603395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.603626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 764/1552] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.604035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.604317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.604729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.604851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fad76b-62ef-4abd-ab3c-cd8777809f49 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.605078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fad76b-62ef-4abd-ab3c-cd8777809f49 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.605348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.605441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fad76b-62ef-4abd-ab3c-cd8777809f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.607263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.626069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d9fad76b-62ef-4abd-ab3c-cd8777809f49 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.627684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d9fad76b-62ef-4abd-ab3c-cd8777809f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.636299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 765/1553] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.640618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 789/1554] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:06.640618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.641416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4375f-97d1-4b0e-936e-e5e30e225cd8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.641833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4375f-97d1-4b0e-936e-e5e30e225cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.642279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4375f-97d1-4b0e-936e-e5e30e225cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.663657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-18f4375f-97d1-4b0e-936e-e5e30e225cd8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.665055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-18f4375f-97d1-4b0e-936e-e5e30e225cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.700028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d96c6-0c04-48aa-b3fe-fd7f0c8c7913 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.704751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.705341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.705663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.706193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.706545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:06.706852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.708489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36221a22-8bbe-4b45-9b18-9ce1012a7823 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.719971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 790/1555] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:06.721309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.721598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.721870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.725510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 766/1556] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.731094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0c96e-9b34-4ec2-9187-a94a2f76d3be None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.731094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0c96e-9b34-4ec2-9187-a94a2f76d3be None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.731094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0c96e-9b34-4ec2-9187-a94a2f76d3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.747808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.748947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.750618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0df0c96e-9b34-4ec2-9187-a94a2f76d3be None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.751802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0df0c96e-9b34-4ec2-9187-a94a2f76d3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.757453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 791/1557] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.758235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.767465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 767/1558] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.767984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4ed05-50e8-4cd9-8a1c-223c46852eb5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.768796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8f65a-6323-4a4b-9833-86f8db059546 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.769062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8f65a-6323-4a4b-9833-86f8db059546 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.769331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8f65a-6323-4a4b-9833-86f8db059546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.773105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.773707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.773988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.774269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.774911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.776557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.786732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 792/1559] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:06.788312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.789684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cee8f65a-6323-4a4b-9833-86f8db059546 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.791159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cee8f65a-6323-4a4b-9833-86f8db059546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.794725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:06.799151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ee678-7ef7-4284-af31-d0c0de151965 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.800336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 768/1560] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.805261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae339c-144e-457b-94e0-8467c92e87fb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.805415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.805556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae339c-144e-457b-94e0-8467c92e87fb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.805804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae339c-144e-457b-94e0-8467c92e87fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.805913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.806116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.806389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.807012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.808104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:06.808104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:58:06.808104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:58:06.809306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.819858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:06.828743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bcae339c-144e-457b-94e0-8467c92e87fb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.830816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bcae339c-144e-457b-94e0-8467c92e87fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.835099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 793/1561] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:06.838930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 769/1562] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.839326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.840201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e59ad2-1023-4481-b55a-a56d40e15025 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.840518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e59ad2-1023-4481-b55a-a56d40e15025 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.840822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e59ad2-1023-4481-b55a-a56d40e15025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.852415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51b030-9592-466d-a186-a7e55527da28 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.857124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.857808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.858157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.858441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.859716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.861421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-05e59ad2-1023-4481-b55a-a56d40e15025 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.861659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.862680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-05e59ad2-1023-4481-b55a-a56d40e15025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.871001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 770/1563] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.872126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.872409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.872686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.889652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 794/1564] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:06.890495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.892993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:06.894211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:06.956014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 771/1565] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:06.957311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.957954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0243390-0c30-4383-84a8-c9b9bdeb8da5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.963095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.963744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.964023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.964309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.964947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:06.965253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.966914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.967480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a05c47-a33a-4364-94bb-ed7814c76bea None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.972283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.972935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.973215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.973492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] PATH_INFO: `/v3/projects/cad002877f904e949722bd742b2d7337/users/e3440f8c4bc44b8f89070b876dec874d/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.978896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 795/1566] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:06.979787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:06.989773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47823e-2007-43ad-b2e9-65999399649a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.994571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.995341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.995626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.995897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.996342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad002877f904e949722bd742b2d7337, user_id=e3440f8c4bc44b8f89070b876dec874d, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.996540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:06.996851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.997600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:06.997600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:58:06.997600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:58:06.998507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.000395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.012136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 796/1567] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:07.014404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 772/1568] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:06 2022] PUT /identity/v3/projects/cad002877f904e949722bd742b2d7337/users/e3440f8c4bc44b8f89070b876dec874d/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:07.020200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.020363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.030820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14655a-b37a-4f9a-8018-af1c22229451 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.083621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f5ea4-3576-48e0-b7aa-fc299ad27460 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.088265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.088432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.088926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.089035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.089151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.089269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.089411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.089552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.090055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:07.090160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.090363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.091733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.091993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.101806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 797/1569] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.102639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.111488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 773/1570] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:07.111488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.112849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961d74-aa48-4005-9f1d-7c0dcfd644ff admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.117613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.118220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.118498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.118771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.119440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:07.119747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.121352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.121769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d3182-0de5-477b-aa41-c64a2f6b0a77 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.127633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.128229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.128746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.129023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.129658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.131657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.134240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 798/1571] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:07.135060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.146169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a3f41-7402-46d2-aaa5-050896a416cd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.151082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.151836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.152117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.152387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.153052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:07.153354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.154945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.155281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 774/1572] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/users => generated 302 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.159037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.174754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 799/1573] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:07.174754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.175046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523df3c2-214d-4191-96f7-9f7c3b8fb709 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.175046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52879eb-c0e9-4815-88b5-1123f1c55a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.175046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52879eb-c0e9-4815-88b5-1123f1c55a2b admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.175046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52879eb-c0e9-4815-88b5-1123f1c55a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.175498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52879eb-c0e9-4815-88b5-1123f1c55a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.175498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52879eb-c0e9-4815-88b5-1123f1c55a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.177086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52879eb-c0e9-4815-88b5-1123f1c55a2b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.178888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1011b3-553e-48ff-a5b4-b2dc8c2d0a64 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.184597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.185187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.185467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.185742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.186377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.189038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.206975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 775/1574] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.207249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:07.207371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.207467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.207773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.224813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 800/1575] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.224813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.237295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] MFA Rules not processed for user `e3440f8c4bc44b8f89070b876dec874d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:07.238519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:07.251949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:07.288289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62b556-3ff0-4d5c-a603-bdec1cda1d18 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.292958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.293664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.293865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.294135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.294773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.296912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.306431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 776/1576] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:07.307176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 801/1577] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.307396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.310872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.320596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82a0e-36aa-4075-a462-7ed9d8178986 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.321400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0b3e-bd94-4bc6-a5e5-49c50897962c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.325157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.325775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.325955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.326063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.326312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.326547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.326827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.326927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.327102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.327792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.329048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.329724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.356582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 777/1578] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.357672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.361890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 802/1579] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.361890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.423863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a3cde-463d-425a-b851-3646d2d312e9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.424332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdf111-1e31-4fc0-8a8b-916ceb79917d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.428568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.429069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.429286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.429560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.429701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.429826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.429978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.430249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.430459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.430878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.432133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.432917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.442765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 778/1580] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.443599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.454015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798e397-6537-4eda-896a-c8f05754ab83 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.458774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.459415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.459699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.459971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.460593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.461100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 803/1581] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.461930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.462585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.486085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 779/1582] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.487025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.496586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2bed4-81e2-46e4-9a59-ecb69ee626f5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.517735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:07.532096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d3a19-7183-49e0-b657-038dd601afa2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.539702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.540315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.540594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.541218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.541868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.544359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.558470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.559272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.559595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.559913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] PATH_INFO: `/v3/projects/6edd1ecc737c48c3b26fa7dea7a48ddd/users/dab1b8b77cfb479098ac9e0d785a25c6/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.571879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 804/1583] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.571879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.576977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6edd1ecc737c48c3b26fa7dea7a48ddd, user_id=dab1b8b77cfb479098ac9e0d785a25c6, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.579749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.582011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:07.595499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 780/1584] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:07 2022] PUT /identity/v3/projects/6edd1ecc737c48c3b26fa7dea7a48ddd/users/dab1b8b77cfb479098ac9e0d785a25c6/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:07.599932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.632059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387fb5f-45c2-41a6-b534-d672cec30c40 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.663110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4306b-9809-4207-a050-daaa11463ffd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.668126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.668902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.669284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.669795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.670435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.673065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.682106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.682761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.683042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.683371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.695352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 781/1585] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.696201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.705932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366e58f-3a2c-495e-b5da-4ed22a2a9eb6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.710825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.711439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.711720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.712014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.712629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.714528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.738527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 782/1586] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.739413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.744365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.745815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.748899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd754396-a349-4e70-9dab-d02c7613bea1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.753916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.754608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.754898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.755006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 805/1587] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:07.755174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.755816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.755926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.757430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.766885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e139f0-44c2-41d4-a8fb-4a9f6c96bac8 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.769436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 783/1588] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.770765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.774791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49b37d-0f81-423a-9009-e961bbac10a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.775652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49b37d-0f81-423a-9009-e961bbac10a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.776095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49b37d-0f81-423a-9009-e961bbac10a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.776527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49b37d-0f81-423a-9009-e961bbac10a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.777478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49b37d-0f81-423a-9009-e961bbac10a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.780094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49b37d-0f81-423a-9009-e961bbac10a2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.781966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420d81c-d06a-4f5c-86fe-a7e878eaffa8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.786707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.787346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.787626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.787914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.788530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.790116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.802021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 784/1589] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.802848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.807321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 806/1590] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/users => generated 294 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:07.808633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.808980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.809745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.813168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcc9a0-5ad3-4d59-a020-5df0e6b6e123 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.817817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.818406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.818691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.818962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.819659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.821246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.832561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:07.832974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 785/1591] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.833866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.834670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:07.844813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 807/1592] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:07.850490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03c7bd-0730-418d-bebb-0941a8aebe90 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.854757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.855072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.855708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.855995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.856256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.856891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.858477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.867016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2bdb5-becc-47c6-8f8f-cec86980515f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.872652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 786/1593] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.872652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.875445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.876142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.876497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.876843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.877573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.880171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.882623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34456bca-abc4-41ea-a38f-ac0e5dba0ffc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.887678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.888325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.888601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.888935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] PATH_INFO: `/v3/projects/5869821fc42c4a7ea595800371029c56/users/00520eff46e242b192aacdb43f0827ff/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.896330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 808/1594] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.896330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.907337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f12e-c616-4d43-bbd5-f0b283de9699 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.909159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5869821fc42c4a7ea595800371029c56, user_id=00520eff46e242b192aacdb43f0827ff, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.912192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.912434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.913135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.913483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.913858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.914926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.917179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.926808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 787/1595] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:07 2022] PUT /identity/v3/projects/5869821fc42c4a7ea595800371029c56/users/00520eff46e242b192aacdb43f0827ff/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:07.927341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.933496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 809/1596] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:07.937530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.949297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfedaf7-1243-4875-91a1-3b1cb532ade6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.953537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:07.996393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:08.000622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5117592-9ebc-4f47-8b5a-d68d165863fd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.007736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.008424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.008864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.009107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.009815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.011844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.022704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.022903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 788/1597] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.026680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.026824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.026943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.026943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.026943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.026943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.038655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 810/1598] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.040608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.051369 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97d39c-d039-4515-9bc9-565105ec6337 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.057487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.058167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.058458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.058733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.059598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.062388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.077419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 811/1599] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.077419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.089029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102db84-90ab-41b3-9e39-ebd4704af03c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.094734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd92169-03d0-4506-b5a8-e898a2393e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.095496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92169-03d0-4506-b5a8-e898a2393e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.095773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92169-03d0-4506-b5a8-e898a2393e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.096055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92169-03d0-4506-b5a8-e898a2393e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.096804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd92169-03d0-4506-b5a8-e898a2393e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.097733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e13770-aa9c-401c-91cd-b08b18f2bbdf admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.098506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd92169-03d0-4506-b5a8-e898a2393e56 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.101635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.102469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.102792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.103129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.113190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 812/1600] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.113190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.113190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.113190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.113958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.115600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.127107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 789/1601] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:08.127979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.142850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be45c8-959c-43f8-8e73-bee72231df3a service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.144042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.145923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.151026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.152632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.153004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.153286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] PATH_INFO: `/v3/projects/35d51e9cc4d343d487cf45c2051c11f2/users/0b4e842eed494513a2d1c3f86d6c3030/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.164229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 813/1602] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:08.164229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.179729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bb1d6-8a7a-4892-9f7f-e11829aa213e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.184629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d51e9cc4d343d487cf45c2051c11f2, user_id=0b4e842eed494513a2d1c3f86d6c3030, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.186316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.187382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.187912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.188536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.189370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.189686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.191563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.205062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 814/1603] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.206432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.214256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 790/1604] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:08 2022] PUT /identity/v3/projects/35d51e9cc4d343d487cf45c2051c11f2/users/0b4e842eed494513a2d1c3f86d6c3030/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:08.215866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.230945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c744-37be-4338-9a71-e9627fb4f414 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.314221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584b9ed-7bae-4547-9267-bcb07bedd54a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.321628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.322318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.322694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.323162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] PATH_INFO: `/v3/projects/3e3eb7b7c1be4710b1cdea1e2dc4bf91/users/48f744a11c594df98d1cbf5fa265d3c2/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.345916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3eb7b7c1be4710b1cdea1e2dc4bf91, user_id=48f744a11c594df98d1cbf5fa265d3c2, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.351039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.366683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.367282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.367645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.367818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] PATH_INFO: `/v3/projects/8194ac89d7394e4f9193411d8bce66fb/users/8b6ffa903cf64bd6b78648dad1f53839/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.374070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 791/1605] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:08 2022] PUT /identity/v3/projects/3e3eb7b7c1be4710b1cdea1e2dc4bf91/users/48f744a11c594df98d1cbf5fa265d3c2/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:08.375846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.400844 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8194ac89d7394e4f9193411d8bce66fb, user_id=8b6ffa903cf64bd6b78648dad1f53839, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.411676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.429809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 815/1606] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:08 2022] PUT /identity/v3/projects/8194ac89d7394e4f9193411d8bce66fb/users/8b6ffa903cf64bd6b78648dad1f53839/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:08.431412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.442759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.498567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:08.509106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820190b-b624-4286-98c4-81900cd570d7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.524409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:08.532328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32591055-083c-44de-be1a-1adff9d92baf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.538327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e353a-17e5-48c9-8884-07006ffe45b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.540491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e353a-17e5-48c9-8884-07006ffe45b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.540943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e353a-17e5-48c9-8884-07006ffe45b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.541776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e353a-17e5-48c9-8884-07006ffe45b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.586781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.587649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.587938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.588215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] PATH_INFO: `/v3/projects/8fd1dbe224fe4121aad305172e9132a3/users/b8f7f53e4f3645f28540cd7b8f78938d/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.607968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd1dbe224fe4121aad305172e9132a3, user_id=b8f7f53e4f3645f28540cd7b8f78938d, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.612043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.630647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 792/1607] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:08 2022] PUT /identity/v3/projects/8fd1dbe224fe4121aad305172e9132a3/users/b8f7f53e4f3645f28540cd7b8f78938d/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:08.631568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.655320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e353a-17e5-48c9-8884-07006ffe45b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.657403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e353a-17e5-48c9-8884-07006ffe45b3 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.668806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 816/1608] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:08.670133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a4e69-5521-4d03-82dd-03e2a3fa8e6a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.670417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a4e69-5521-4d03-82dd-03e2a3fa8e6a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.670716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a4e69-5521-4d03-82dd-03e2a3fa8e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.701267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a67a4e69-5521-4d03-82dd-03e2a3fa8e6a None None] MFA Rules not processed for user `0b4e842eed494513a2d1c3f86d6c3030`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.702544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a67a4e69-5521-4d03-82dd-03e2a3fa8e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.733679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad157de-0167-4917-ab36-3f16968b6caa admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.739836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334afbe-7d4d-4993-b07e-fb10df0adc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.740806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334afbe-7d4d-4993-b07e-fb10df0adc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.741100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334afbe-7d4d-4993-b07e-fb10df0adc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.741417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334afbe-7d4d-4993-b07e-fb10df0adc9b admin admin] PATH_INFO: `/v3/projects/58581ef3e3844816923467523d5be072/users/2febff9d4cbb476c90037ffacf44465d/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.771183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334afbe-7d4d-4993-b07e-fb10df0adc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58581ef3e3844816923467523d5be072, user_id=2febff9d4cbb476c90037ffacf44465d, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.774588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334afbe-7d4d-4993-b07e-fb10df0adc9b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.785537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 817/1609] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:08.789024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77039f-4f46-44ed-9f0b-81afd4f9351e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.789380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77039f-4f46-44ed-9f0b-81afd4f9351e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.789723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77039f-4f46-44ed-9f0b-81afd4f9351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.790235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 793/1610] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:08 2022] PUT /identity/v3/projects/58581ef3e3844816923467523d5be072/users/2febff9d4cbb476c90037ffacf44465d/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:08.791581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.792126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.792457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.814322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-eb77039f-4f46-44ed-9f0b-81afd4f9351e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.815924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eb77039f-4f46-44ed-9f0b-81afd4f9351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.834724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] MFA Rules not processed for user `8b6ffa903cf64bd6b78648dad1f53839`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.836000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.889611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 818/1611] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:08.891625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.892104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.892477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.915283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:08.923859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 794/1612] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:08.925602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.932495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] MFA Rules not processed for user `b8f7f53e4f3645f28540cd7b8f78938d`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.934513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.942725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601dad7-1aab-48de-ab11-046613ba809f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.948610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76426-df9d-4f24-bea7-43690a4eef56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.949227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76426-df9d-4f24-bea7-43690a4eef56 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.949499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76426-df9d-4f24-bea7-43690a4eef56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.949772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76426-df9d-4f24-bea7-43690a4eef56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.950415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76426-df9d-4f24-bea7-43690a4eef56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.952462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76426-df9d-4f24-bea7-43690a4eef56 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.953847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:08.985508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 795/1613] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:08.985508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.985508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.985508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.008576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:09.009821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:09.022439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 819/1614] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:09.022439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.101222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedfe5d-d23c-47a6-85c6-a375327a7906 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.102776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 796/1615] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:09.104053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.118806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab8928-a7ad-4f08-8f14-3ea6a3041d73 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.124549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.125158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.125434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.125700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.126769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.129144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.163492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 797/1616] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:09.164669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.173942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8bcab-6057-4da7-b133-801c8ee618a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.175672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bcab-6057-4da7-b133-801c8ee618a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.176106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bcab-6057-4da7-b133-801c8ee618a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.176674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bcab-6057-4da7-b133-801c8ee618a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.192742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8bcab-6057-4da7-b133-801c8ee618a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.195017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8bcab-6057-4da7-b133-801c8ee618a4 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.209995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 820/1617] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:09.211545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985caca-ac21-418e-9e6c-57e6e30ec0a1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.211940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985caca-ac21-418e-9e6c-57e6e30ec0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.212575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985caca-ac21-418e-9e6c-57e6e30ec0a1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.215110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 821/1618] 10.222.0.66 () {54 vars in 961 bytes} [Thu Sep 22 14:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:09.221855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985caca-ac21-418e-9e6c-57e6e30ec0a1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.251441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60c6e-ed25-435a-a70a-6662f4fb01b7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.258486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.259145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.260812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.261120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.261822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.263962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.293538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 798/1619] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:09.294499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.324124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4eb58-9022-486b-92c1-358f2eac02ba admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.338555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dc1a5-8529-4c47-930b-1fe6f51b23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.339562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dc1a5-8529-4c47-930b-1fe6f51b23b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.339757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dc1a5-8529-4c47-930b-1fe6f51b23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.340023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dc1a5-8529-4c47-930b-1fe6f51b23b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.340951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dc1a5-8529-4c47-930b-1fe6f51b23b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.343735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39dc1a5-8529-4c47-930b-1fe6f51b23b5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.352876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985caca-ac21-418e-9e6c-57e6e30ec0a1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.358027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.358871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.359839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.360117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.374626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.377057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.387068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 799/1620] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:09.390876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 822/1621] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:09.391282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.391574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.391765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.397560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.410065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5e4ad-3c9a-49f6-bd26-a85f460a48c9 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.425481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.426711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.427407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.429632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.432452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.440080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.464783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 823/1622] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:09.474237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.499976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:09.502166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b90e13-eb35-4e8f-9214-75665fb5e606 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.503246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:09.527633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.527984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.529941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.531727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.534090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.536799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:09.537674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.544273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:09.579692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 824/1623] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:09.583483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.605161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f91325-c2fe-4edc-affb-fd2c75ade596 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.620094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.621023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.621306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.621583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] PATH_INFO: `/v3/projects/63f87bdff23040d0b295d34cbaf800ba/users/6e958ad9a21b4cc884dcd33295037193/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.654906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f87bdff23040d0b295d34cbaf800ba, user_id=6e958ad9a21b4cc884dcd33295037193, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.661735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.689681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 825/1624] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:09 2022] PUT /identity/v3/projects/63f87bdff23040d0b295d34cbaf800ba/users/6e958ad9a21b4cc884dcd33295037193/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:09.692270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.693300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 800/1625] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:09.694418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.715557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bf9bb-715f-42bb-8990-9826566aec18 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.827554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.828630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.828770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.831100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.833063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.837999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.857087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b3cc6-9a2a-4a02-b2df-7a67e788c5c1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.867246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 801/1626] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:09.868080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.870617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.871334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.871675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.871960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] PATH_INFO: `/v3/projects/12d5a7195d9f4a468e45982332697627/users/e63a8394bd82448ca84ef78ee92362f3/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.902835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d5a7195d9f4a468e45982332697627, user_id=e63a8394bd82448ca84ef78ee92362f3, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.908924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.935245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 826/1627] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:09 2022] PUT /identity/v3/projects/12d5a7195d9f4a468e45982332697627/users/e63a8394bd82448ca84ef78ee92362f3/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:09.938171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.062965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c68a2-c3c2-4d7a-b7e7-84837e208bad admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.075394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3395e2-2dea-4d13-b8b0-1b211ce2df72 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.088862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.090176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.090458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.090786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] PATH_INFO: `/v3/projects/63f87bdff23040d0b295d34cbaf800ba/users/6e958ad9a21b4cc884dcd33295037193/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.098373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f87bdff23040d0b295d34cbaf800ba, user_id=6e958ad9a21b4cc884dcd33295037193, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.104655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.127380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 827/1628] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:09 2022] PUT /identity/v3/projects/63f87bdff23040d0b295d34cbaf800ba/users/6e958ad9a21b4cc884dcd33295037193/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:10.127380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.157938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:10.216581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.219914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.220722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.221747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.248090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:10.276842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deec63-6811-46c1-9f3c-16503840003f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.283000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6a5cc-735a-4239-a37f-687ba9663d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.284168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6a5cc-735a-4239-a37f-687ba9663d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.285210 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6a5cc-735a-4239-a37f-687ba9663d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.285210 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6a5cc-735a-4239-a37f-687ba9663d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.285905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6a5cc-735a-4239-a37f-687ba9663d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.287720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6a5cc-735a-4239-a37f-687ba9663d49 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.302514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 828/1629] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:10.304029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b8fcb-2b13-4a5a-a971-5208ff552853 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.304516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b8fcb-2b13-4a5a-a971-5208ff552853 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.305405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b8fcb-2b13-4a5a-a971-5208ff552853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.333816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.337176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.346700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6a4b8fcb-2b13-4a5a-a971-5208ff552853 None None] MFA Rules not processed for user `6e958ad9a21b4cc884dcd33295037193`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:10.349199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6a4b8fcb-2b13-4a5a-a971-5208ff552853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:10.352186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 802/1630] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:10.353584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.368610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf22af-241d-4299-b9f3-a29f7a8218d3 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.375980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5c685-7f23-44c4-8af1-6f404d75b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.377433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5c685-7f23-44c4-8af1-6f404d75b648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.378483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5c685-7f23-44c4-8af1-6f404d75b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.378977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5c685-7f23-44c4-8af1-6f404d75b648 admin admin] PATH_INFO: `/v3/projects/12d5a7195d9f4a468e45982332697627/users/e63a8394bd82448ca84ef78ee92362f3/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.382736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5c685-7f23-44c4-8af1-6f404d75b648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d5a7195d9f4a468e45982332697627, user_id=e63a8394bd82448ca84ef78ee92362f3, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.388554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5c685-7f23-44c4-8af1-6f404d75b648 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.410545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 803/1631] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:10 2022] PUT /identity/v3/projects/12d5a7195d9f4a468e45982332697627/users/e63a8394bd82448ca84ef78ee92362f3/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:10.422421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.423113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.434442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.456631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 829/1632] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:10.458304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.458756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.459401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.480042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] MFA Rules not processed for user `e63a8394bd82448ca84ef78ee92362f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:10.482081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:10.507712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] MFA Rules not processed for user `00520eff46e242b192aacdb43f0827ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:10.509356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:10.568587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 804/1633] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:10.569940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.613560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 830/1634] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:10.615231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.659013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:10.674719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03fe7-c1bb-4507-af6d-a907162396a1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.684805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d3e20-79c4-4da3-b964-b1d9c8a664d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.687194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d3e20-79c4-4da3-b964-b1d9c8a664d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.691222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d3e20-79c4-4da3-b964-b1d9c8a664d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.691977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d3e20-79c4-4da3-b964-b1d9c8a664d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.729327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4986c-07db-4dee-814c-77ec1ec96b1e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.733175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.734483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.734917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.735429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.746314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.748484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.773596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 831/1635] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:10.774778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.789938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:10.824925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d3e20-79c4-4da3-b964-b1d9c8a664d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.834753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d3e20-79c4-4da3-b964-b1d9c8a664d6 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.852328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 805/1636] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:10.961290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdc431-55dd-4252-a520-6e6e8d3a26d2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.961735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdc431-55dd-4252-a520-6e6e8d3a26d2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.962112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdc431-55dd-4252-a520-6e6e8d3a26d2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.963668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 806/1637] 10.222.0.66 () {54 vars in 912 bytes} [Thu Sep 22 14:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:10.976826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.977212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.978151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.987135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff6d39-dea6-41f8-bf49-76c64b2b1bd4 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.013194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:11.014041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.014744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.015598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.015917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.016561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.016684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:11.018740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.118289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 832/1638] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:10 2022] POST /identity/v3/projects => generated 405 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:11.119219 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.176607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 807/1639] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 22 14:58:11.177657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.197907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfced909-9f18-4d0b-b725-77783b1a8210 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.242802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b6329-c48b-43b2-bc8b-794f3d517785 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.257136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.257706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.257991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.258261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.313563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f40aa9-b590-42d3-8f29-7fccb7991ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.314402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40aa9-b590-42d3-8f29-7fccb7991ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.314834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40aa9-b590-42d3-8f29-7fccb7991ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.315239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f40aa9-b590-42d3-8f29-7fccb7991ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.412824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.414315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.439484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:11.448027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 833/1640] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:11.449020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.486098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f40aa9-b590-42d3-8f29-7fccb7991ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.495982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f40aa9-b590-42d3-8f29-7fccb7991ef9 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.527834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 808/1641] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:11.619994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:11.751069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d1ce8-72a2-42ab-8517-80df1ce62c31 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.751069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.751069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.751069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.752263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.752263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.752263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.752685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750bcd5-67da-4d4d-9913-8e4fed9b11c2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.752685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750bcd5-67da-4d4d-9913-8e4fed9b11c2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.752685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750bcd5-67da-4d4d-9913-8e4fed9b11c2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.752685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 809/1642] 10.222.0.66 () {54 vars in 961 bytes} [Thu Sep 22 14:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:58:11.771574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.771971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.772395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.783010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 834/1643] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:11 2022] POST /identity/v3/users => generated 309 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:11.787259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.805288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd636c-04af-4e70-bf66-3cdb1016c712 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.809863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.810526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.810853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.811134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.815603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:11.819412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:11.901959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.903467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.924898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 835/1644] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:11.925599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.948001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe02de5-1d1c-4e88-83f0-6a00ce73c259 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.968070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762eee-3f7b-41b1-afdb-827bdeb2bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.969231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762eee-3f7b-41b1-afdb-827bdeb2bcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.969506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762eee-3f7b-41b1-afdb-827bdeb2bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.969897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762eee-3f7b-41b1-afdb-827bdeb2bcf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.970654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762eee-3f7b-41b1-afdb-827bdeb2bcf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.972681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762eee-3f7b-41b1-afdb-827bdeb2bcf8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.989498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 810/1645] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Sep 22 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:11.991954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 836/1646] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:11.997142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:12.003726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c66e-c3e6-4cc0-b5fa-3d9615ffde9a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:12.004015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c66e-c3e6-4cc0-b5fa-3d9615ffde9a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:12.004289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c66e-c3e6-4cc0-b5fa-3d9615ffde9a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:12.005786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 837/1647] 10.222.0.66 () {54 vars in 912 bytes} [Thu Sep 22 14:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:58:12.011639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:12.012015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:12.012288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:12.020959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946a31-9432-4e98-b054-4932e4088cdf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:12.033844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:12.034749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:12.035148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:12.035590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] PATH_INFO: `/v3/projects/4b5a721a12ed425baf3ec4e355d68833/users/ded2ffca13b5404090e39b1ce42b4920/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:12.065609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:12.075070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:12.075550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5a721a12ed425baf3ec4e355d68833, user_id=ded2ffca13b5404090e39b1ce42b4920, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:12.078571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:12.110084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 811/1648] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:11 2022] PUT /identity/v3/projects/4b5a721a12ed425baf3ec4e355d68833/users/ded2ffca13b5404090e39b1ce42b4920/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:12.113902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:12.122509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 838/1649] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:12.268926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92d3a6-9ccd-4e6e-b66e-806c6c683a6a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:12.276432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:12.277690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:12.277983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:12.278434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:12.279161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:12.281432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:12.298877 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 812/1650] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:12.302894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:12.325497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c2c52-0d39-4ab8-942b-6c62902c8c5b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:12.335793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:12.340567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e07a6-6b91-42c7-9360-7dc96ac1953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:12.341828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e07a6-6b91-42c7-9360-7dc96ac1953a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:12.343170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e07a6-6b91-42c7-9360-7dc96ac1953a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:12.344211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e07a6-6b91-42c7-9360-7dc96ac1953a admin admin] PATH_INFO: `/v3/projects/4b5a721a12ed425baf3ec4e355d68833/users/ded2ffca13b5404090e39b1ce42b4920/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:12.349196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e07a6-6b91-42c7-9360-7dc96ac1953a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5a721a12ed425baf3ec4e355d68833, user_id=ded2ffca13b5404090e39b1ce42b4920, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:12.356978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e07a6-6b91-42c7-9360-7dc96ac1953a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:12.411278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 839/1651] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:12 2022] PUT /identity/v3/projects/4b5a721a12ed425baf3ec4e355d68833/users/ded2ffca13b5404090e39b1ce42b4920/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:12.492151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:12.505728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8f77d-ad8b-4c49-b5c9-2c55dfd2dcdf admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:12.631289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e686b11-b6a6-488c-8145-e9652d7fddde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:12.631930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686b11-b6a6-488c-8145-e9652d7fddde service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:12.632211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686b11-b6a6-488c-8145-e9652d7fddde service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:12.632488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e686b11-b6a6-488c-8145-e9652d7fddde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:12.840370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686b11-b6a6-488c-8145-e9652d7fddde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:12.840370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e686b11-b6a6-488c-8145-e9652d7fddde service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:12.840370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 813/1652] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:15.794284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400dc6-f471-46ce-9918-5d3e6a7f1173 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:15.795732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400dc6-f471-46ce-9918-5d3e6a7f1173 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:15.796694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400dc6-f471-46ce-9918-5d3e6a7f1173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:15.858283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c5400dc6-f471-46ce-9918-5d3e6a7f1173 None None] MFA Rules not processed for user `ded2ffca13b5404090e39b1ce42b4920`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:15.862285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c5400dc6-f471-46ce-9918-5d3e6a7f1173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:16.040885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 840/1653] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:17.680763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec12fdf-5f73-4db7-b177-7693f761876a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.681814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec12fdf-5f73-4db7-b177-7693f761876a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.682306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec12fdf-5f73-4db7-b177-7693f761876a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.686040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 814/1654] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:58:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:17.745585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.746557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.746837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.786408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:17.789510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.805091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:17.989835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 841/1655] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:18.050625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec12fdf-5f73-4db7-b177-7693f761876a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:18.091732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec12fdf-5f73-4db7-b177-7693f761876a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:18.110238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5246d5-6a40-4c6e-be98-e797b11c883d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:18.111184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5246d5-6a40-4c6e-be98-e797b11c883d service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.111626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5246d5-6a40-4c6e-be98-e797b11c883d service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.115048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5246d5-6a40-4c6e-be98-e797b11c883d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.147852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5246d5-6a40-4c6e-be98-e797b11c883d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:18.149468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5246d5-6a40-4c6e-be98-e797b11c883d service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:18.198405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 815/1656] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:20.026126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.220918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5b9b7-ae4e-4ddb-85a3-e8ec9946059d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.238769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.239489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.239772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.240046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] PATH_INFO: `/v3/users/00520eff46e242b192aacdb43f0827ff` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.241216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] RBAC: Authorizing `identity:delete_user(user_id=00520eff46e242b192aacdb43f0827ff)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.243215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.332523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00520eff46e242b192aacdb43f0827ff'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:20.346899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00520eff46e242b192aacdb43f0827ff'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:20.386099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00520eff46e242b192aacdb43f0827ff'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:20.387485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00520eff46e242b192aacdb43f0827ff'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:20.388153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a1cadf8f-4061-4fff-a696-d9ce1c43399d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00520eff46e242b192aacdb43f0827ff'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:20.418673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 842/1657] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:20 2022] DELETE /identity/v3/users/00520eff46e242b192aacdb43f0827ff => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:20.520553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.521514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.521731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.571163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] MFA Rules not processed for user `48f744a11c594df98d1cbf5fa265d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.572438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.711392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 816/1658] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:20.721654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508eb09-b93f-4fe2-8dd7-fae526b2333b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.722167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508eb09-b93f-4fe2-8dd7-fae526b2333b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.722727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508eb09-b93f-4fe2-8dd7-fae526b2333b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.763507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2508eb09-b93f-4fe2-8dd7-fae526b2333b None None] MFA Rules not processed for user `48f744a11c594df98d1cbf5fa265d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.766325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2508eb09-b93f-4fe2-8dd7-fae526b2333b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.791960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 843/1659] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:20.803240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.804821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.850658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:20.899028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06f133-8712-45de-a61d-9cc7b6ac4753 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.899543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06f133-8712-45de-a61d-9cc7b6ac4753 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.899922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06f133-8712-45de-a61d-9cc7b6ac4753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.954279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3a06f133-8712-45de-a61d-9cc7b6ac4753 None None] MFA Rules not processed for user `dab1b8b77cfb479098ac9e0d785a25c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.956784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3a06f133-8712-45de-a61d-9cc7b6ac4753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.965141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462cb46-f26c-4aa5-829c-ec97be6ee68b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.972813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:20.991960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.992608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.992912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.993184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.036647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.038270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.059046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 817/1660] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:21.059970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.064161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 844/1661] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:21.077147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.077690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:21.078054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.078528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.106664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] MFA Rules not processed for user `dab1b8b77cfb479098ac9e0d785a25c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:21.108291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:21.119095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 845/1662] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:21.133891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.136859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:21.159721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8c78a-87ef-43f4-9da9-bf7fababc390 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.169029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.169985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.172049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.172815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.185208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.188604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.218732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 846/1663] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:21.229948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7de24d-6a00-4e2e-afc7-a501574796f0 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.240560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.241638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.242178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.242978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] PATH_INFO: `/v3/projects/5869821fc42c4a7ea595800371029c56` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.261279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5869821fc42c4a7ea595800371029c56)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.263814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.435369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5869821fc42c4a7ea595800371029c56', 'request_id': 'req-60605be6-9d41-4a48-b1cb-f54c5d539be4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.465349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5869821fc42c4a7ea595800371029c56', 'request_id': 'req-60605be6-9d41-4a48-b1cb-f54c5d539be4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.492874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5869821fc42c4a7ea595800371029c56', 'request_id': 'req-60605be6-9d41-4a48-b1cb-f54c5d539be4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.520816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] The token cache is being invalidate because project 5869821fc42c4a7ea595800371029c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:21.521404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.523252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-60605be6-9d41-4a48-b1cb-f54c5d539be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.525388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 818/1664] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:21 2022] DELETE /identity/v3/projects/5869821fc42c4a7ea595800371029c56 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:21.531175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.546221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b420de1-0685-4f59-8daa-669f437118bf service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.715235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.717056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.717421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.717877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] PATH_INFO: `/v3/users/ded2ffca13b5404090e39b1ce42b4920` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.720504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded2ffca13b5404090e39b1ce42b4920)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.723379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.812806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded2ffca13b5404090e39b1ce42b4920'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.842225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded2ffca13b5404090e39b1ce42b4920'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.886965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded2ffca13b5404090e39b1ce42b4920'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.888256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded2ffca13b5404090e39b1ce42b4920'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.890443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e66daa88-0189-480c-823d-c382bbedb1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded2ffca13b5404090e39b1ce42b4920'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:21.943522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 847/1665] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:21 2022] DELETE /identity/v3/users/ded2ffca13b5404090e39b1ce42b4920 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:22.323586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d768a-5d74-436a-903b-1f9d464175d3 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.324235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d768a-5d74-436a-903b-1f9d464175d3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.324725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d768a-5d74-436a-903b-1f9d464175d3 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.326338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 819/1666] 10.222.0.66 () {54 vars in 912 bytes} [Thu Sep 22 14:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:22.333682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.334742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.335329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.344474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3157c6-6349-4f0c-b0d6-cc82835a4cc3 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.345042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3157c6-6349-4f0c-b0d6-cc82835a4cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.345404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3157c6-6349-4f0c-b0d6-cc82835a4cc3 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.349303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 820/1667] 10.222.0.66 () {54 vars in 912 bytes} [Thu Sep 22 14:58:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:22.373768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.374739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.375090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.402534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:22.474677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] MFA Rules not processed for user `60c809b5a47a4983be26116553934387`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:22.476634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.485000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] MFA Rules not processed for user `60c809b5a47a4983be26116553934387`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:22.486255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.525054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:22.629140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 848/1668] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:22.630311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 821/1669] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:22.636745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.638768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.641281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.647342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.654006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fe096-b3f9-4483-91ab-1a6f57e836a6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.660854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.665187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.666441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.667179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.681116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.683029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.697309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 822/1670] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:22.699423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.730944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d54b-9268-46c2-a48b-d8699f0c0622 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.736646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fd545-d3b9-4fd3-8d80-cbf17a5a25b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.737676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd545-d3b9-4fd3-8d80-cbf17a5a25b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.737785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd545-d3b9-4fd3-8d80-cbf17a5a25b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.738002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd545-d3b9-4fd3-8d80-cbf17a5a25b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.746436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fd545-d3b9-4fd3-8d80-cbf17a5a25b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.748183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fd545-d3b9-4fd3-8d80-cbf17a5a25b7 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.769801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 823/1671] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:22.806934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb81fc-2168-4045-9da3-c80beb36a918 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.814144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.815105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.816017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.816475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] PATH_INFO: `/v3/projects/4b5a721a12ed425baf3ec4e355d68833` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.818506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5a721a12ed425baf3ec4e355d68833)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.820876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:22.820876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 22 14:58:22.820876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: Sep 22 14:58:22.822043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.963146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5a721a12ed425baf3ec4e355d68833', 'request_id': 'req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:22.985722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5a721a12ed425baf3ec4e355d68833', 'request_id': 'req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:23.002672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ac491-7dce-453d-87c9-baa62f9d90b0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.002983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ac491-7dce-453d-87c9-baa62f9d90b0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.003514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ac491-7dce-453d-87c9-baa62f9d90b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.011674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5a721a12ed425baf3ec4e355d68833', 'request_id': 'req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:23.067801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-284ac491-7dce-453d-87c9-baa62f9d90b0 None None] MFA Rules not processed for user `2febff9d4cbb476c90037ffacf44465d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:23.068891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] The token cache is being invalidate because project 4b5a721a12ed425baf3ec4e355d68833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:23.070077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:23.072133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-284ac491-7dce-453d-87c9-baa62f9d90b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:23.073649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4add84de-a66e-4ce0-8ce4-b67bcb8792fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:23.075964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 849/1672] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:22 2022] DELETE /identity/v3/projects/4b5a721a12ed425baf3ec4e355d68833 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:23.082795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.083195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.084455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.130890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:23.132278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:23.204777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 824/1673] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:23.236784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:23.294501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 850/1674] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:23.304269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.304774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.305859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.363565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:23.365677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:23.398579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 825/1675] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:23.420887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.440115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fbdd4-8410-45cb-ab26-93f471c7e19d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.463038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.469178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.469483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.469764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.470450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:23.470765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:23.475893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:23.520116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 851/1676] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:23.522007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.536822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd57d-ca23-47d3-bf32-35e95c439fa4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.551991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.552627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.552906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.553503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.554321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:23.560199 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:23.640780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 826/1677] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:23.645025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.658267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeca36-5b18-4c81-8d8a-7093630bd3df admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.715734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.718640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:23.881524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.882228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.882577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.882926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.884522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:23.889992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:23.927950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd643a5-6ba9-4891-9b11-71ac2450d144 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.935485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df931e0-7642-46f9-bf67-a0894911d65e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.936376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df931e0-7642-46f9-bf67-a0894911d65e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.937258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df931e0-7642-46f9-bf67-a0894911d65e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.938148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df931e0-7642-46f9-bf67-a0894911d65e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.947080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:23.964791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 852/1678] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/users => generated 282 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:23.965665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.974056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.038361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df931e0-7642-46f9-bf67-a0894911d65e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.042032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df931e0-7642-46f9-bf67-a0894911d65e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.054774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 827/1679] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:24.070778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc094b9-c091-4878-bc73-3779db71e3d7 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.071704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc094b9-c091-4878-bc73-3779db71e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.071989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc094b9-c091-4878-bc73-3779db71e3d7 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.074037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 828/1680] 10.222.0.66 () {54 vars in 961 bytes} [Thu Sep 22 14:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:24.074874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc094b9-c091-4878-bc73-3779db71e3d7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.106330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc094b9-c091-4878-bc73-3779db71e3d7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.122521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e21e3-2da5-4bf8-ad8e-36d2c5bef26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.123147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e21e3-2da5-4bf8-ad8e-36d2c5bef26d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.123453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e21e3-2da5-4bf8-ad8e-36d2c5bef26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.123728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e21e3-2da5-4bf8-ad8e-36d2c5bef26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.130529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e21e3-2da5-4bf8-ad8e-36d2c5bef26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.133442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e21e3-2da5-4bf8-ad8e-36d2c5bef26d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.153105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 829/1681] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:24.162971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.163297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.170843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.250168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:24.251929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.275575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7d11d-2b22-4fae-93da-77b7f88d61b3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.281572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 830/1682] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Sep 22 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:24.283638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.299461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5e5e3-ebb5-4dee-a717-cee12f6a5c4a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.307565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.308019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.308310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.308594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.314704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.315338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5bddb-9ad1-4911-98db-f943cee69be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.316603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5bddb-9ad1-4911-98db-f943cee69be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.316992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5bddb-9ad1-4911-98db-f943cee69be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.317426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5bddb-9ad1-4911-98db-f943cee69be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.318102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5bddb-9ad1-4911-98db-f943cee69be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.319820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5bddb-9ad1-4911-98db-f943cee69be8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.350661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 831/1683] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:24.351967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.355004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.355962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab2b3-5a57-4031-9835-f94b9fadfe6c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.356284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab2b3-5a57-4031-9835-f94b9fadfe6c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.356571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beab2b3-5a57-4031-9835-f94b9fadfe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.368731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 853/1684] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:24.369920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.387418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a42d2d-1663-4645-877c-29dfe01a907e service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.401164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.405226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.405941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.406365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.420164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5beab2b3-5a57-4031-9835-f94b9fadfe6c None None] MFA Rules not processed for user `dab1b8b77cfb479098ac9e0d785a25c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:24.423443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5beab2b3-5a57-4031-9835-f94b9fadfe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.427815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.429322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.460814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 854/1685] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:24.463535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.491237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3dc00-a12a-4839-87ad-7ac5229601c2 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.504504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.505011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.505299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.505580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] PATH_INFO: `/v3/projects/9c15667aecdc4b4e930fdc74aab2fac5/users/766c7db7523e428fbb6762da46ea4286/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.544147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c15667aecdc4b4e930fdc74aab2fac5, user_id=766c7db7523e428fbb6762da46ea4286, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.549735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.566426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 832/1686] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:24.590520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 855/1687] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:24 2022] PUT /identity/v3/projects/9c15667aecdc4b4e930fdc74aab2fac5/users/766c7db7523e428fbb6762da46ea4286/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:24.592565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.593472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.594326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.693605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] MFA Rules not processed for user `766c7db7523e428fbb6762da46ea4286`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:24.696244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.872416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 833/1688] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:24.872862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.883187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:25.013927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd2563-581f-4134-b145-d3f8f43eca59 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.026335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.027027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.028081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.028368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.029017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.035575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.119734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:25.120232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 856/1689] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:24 2022] POST /identity/v3/projects => generated 365 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:25.148075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a58f47-20bf-44d8-89b2-e6f8dfca4c45 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.185421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:25.432043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.433043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.433043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.433224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.433898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.437091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.500474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:25.531320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 834/1690] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:25 2022] POST /identity/v3/users => generated 289 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:25.535121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b13ade-3fa1-47d1-84b2-f0212cb95fda admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.542459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.543457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.543899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.545272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.546712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.548638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.570705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 857/1691] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:25.571104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:25.584039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432966c-f008-4bc5-8545-12f14989c1ca admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.613951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.614846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.615564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.617160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] PATH_INFO: `/v3/projects/acb21b6887f24246ac0341d7882776cd/users/dadd1055f3e84f6abfaa1e475837e1aa/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.676294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb21b6887f24246ac0341d7882776cd, user_id=dadd1055f3e84f6abfaa1e475837e1aa, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.683952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.730256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 835/1692] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:25 2022] PUT /identity/v3/projects/acb21b6887f24246ac0341d7882776cd/users/dadd1055f3e84f6abfaa1e475837e1aa/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:25.736618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:25.755521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca9e92-1250-4820-83a2-57b7d0f56e73 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.952969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5996704-afb0-4e5e-95b2-819e8ddc065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.954717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5996704-afb0-4e5e-95b2-819e8ddc065d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.955111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5996704-afb0-4e5e-95b2-819e8ddc065d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.956462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5996704-afb0-4e5e-95b2-819e8ddc065d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.957451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5996704-afb0-4e5e-95b2-819e8ddc065d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.962370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5996704-afb0-4e5e-95b2-819e8ddc065d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.990547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 858/1693] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:25.994764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:26.011607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d82386-eb54-4eb1-b2bc-66bbe8c7a64b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:26.028692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:26.029915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:26.032791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:26.033332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] PATH_INFO: `/v3/projects/acb21b6887f24246ac0341d7882776cd/users/dadd1055f3e84f6abfaa1e475837e1aa/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:26.037057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb21b6887f24246ac0341d7882776cd, user_id=dadd1055f3e84f6abfaa1e475837e1aa, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:26.050746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:26.074808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 836/1694] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:25 2022] PUT /identity/v3/projects/acb21b6887f24246ac0341d7882776cd/users/dadd1055f3e84f6abfaa1e475837e1aa/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:26.086272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:26.086701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:26.087110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:26.155786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] MFA Rules not processed for user `dadd1055f3e84f6abfaa1e475837e1aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:26.163635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:26.404601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:26.416293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 859/1695] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:26.428932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:26.430870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:26.642055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd2fd5-6b49-4351-ac39-66a2181fc224 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:26.651968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52cf7c-9354-4eae-9680-37214a9cd33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:26.652612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52cf7c-9354-4eae-9680-37214a9cd33c service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:26.652946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52cf7c-9354-4eae-9680-37214a9cd33c service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:26.653222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52cf7c-9354-4eae-9680-37214a9cd33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:26.677601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52cf7c-9354-4eae-9680-37214a9cd33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:26.679071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52cf7c-9354-4eae-9680-37214a9cd33c service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:26.702845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 837/1696] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:27.534247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:27.644302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b027d-3a64-4bc0-8590-c464c73e6f4e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:27.651250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:27.652206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:27.652729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:27.653212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] PATH_INFO: `/v3/users/766c7db7523e428fbb6762da46ea4286` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:27.654737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766c7db7523e428fbb6762da46ea4286)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:27.657716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:27.784155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766c7db7523e428fbb6762da46ea4286'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:27.799915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766c7db7523e428fbb6762da46ea4286'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:27.832603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766c7db7523e428fbb6762da46ea4286'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:27.833592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766c7db7523e428fbb6762da46ea4286'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:27.834397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-02cac77b-781d-46f2-8f98-6b7ba143b220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766c7db7523e428fbb6762da46ea4286'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:27.891664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 860/1697] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:27 2022] DELETE /identity/v3/users/766c7db7523e428fbb6762da46ea4286 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:27.905712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd94d-f2b3-4c65-957d-fbc1d8299c2f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:27.906264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:27.906345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd94d-f2b3-4c65-957d-fbc1d8299c2f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:27.906431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:27.906500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd94d-f2b3-4c65-957d-fbc1d8299c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:27.906572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:27.957377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] MFA Rules not processed for user `2febff9d4cbb476c90037ffacf44465d`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:27.960067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:27.980598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a49cd94d-f2b3-4c65-957d-fbc1d8299c2f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:27.981850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a49cd94d-f2b3-4c65-957d-fbc1d8299c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:28.190992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 861/1698] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:28.231016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 838/1699] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:28.242758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:28.244228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:28.310785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:28.353451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac744-e77c-4057-b768-7fa21a87bac5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:28.357113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:28.357889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:28.358688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:28.358964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:28.374808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:28.376866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:28.390799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 862/1700] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:28.886657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:28.886949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:28.887458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:28.923084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:29.061160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:29.065624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:29.242440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 839/1701] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:29.500160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:29.501657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:29.524627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce4279-4595-4dac-8590-47aee83208c5 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:29.535564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:29.538175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:29.538798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:29.539233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] PATH_INFO: `/v3/projects/9c15667aecdc4b4e930fdc74aab2fac5` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:29.541341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c15667aecdc4b4e930fdc74aab2fac5)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:29.546244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:29.559446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:29.560913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:29.659403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c15667aecdc4b4e930fdc74aab2fac5', 'request_id': 'req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:29.671852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c15667aecdc4b4e930fdc74aab2fac5', 'request_id': 'req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:29.686983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c15667aecdc4b4e930fdc74aab2fac5', 'request_id': 'req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:29.734876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] The token cache is being invalidate because project 9c15667aecdc4b4e930fdc74aab2fac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:29.738905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:29.739985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:29.749965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 863/1702] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:29 2022] DELETE /identity/v3/projects/9c15667aecdc4b4e930fdc74aab2fac5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:29.754226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:29.757661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:29.768358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94f4c7-9454-42d4-973b-eb1af26b7247 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:29.927363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:29.939614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:29.940327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:29.941093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:29.941613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:29.944042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:29.972840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5eb8da-6e40-49be-8a1e-f6797bd628eb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:30.001436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:30.005980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:30.006914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:30.007404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] PATH_INFO: `/v3/users/dadd1055f3e84f6abfaa1e475837e1aa` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:30.009035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] RBAC: Authorizing `identity:delete_user(user_id=dadd1055f3e84f6abfaa1e475837e1aa)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:30.013846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:30.019806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:30.021675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:30.033598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 840/1703] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 14:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 14:58:30.322357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dadd1055f3e84f6abfaa1e475837e1aa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:30.336693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:30.348721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dadd1055f3e84f6abfaa1e475837e1aa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:30.398598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dadd1055f3e84f6abfaa1e475837e1aa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:30.399390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dadd1055f3e84f6abfaa1e475837e1aa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:30.400617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dadd1055f3e84f6abfaa1e475837e1aa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:30.458926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 864/1704] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:29 2022] DELETE /identity/v3/users/dadd1055f3e84f6abfaa1e475837e1aa => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:31.197317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:31.198885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:31.222436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052455e-4575-4c22-9279-bc7a82c00802 service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:31.305469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:31.308018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:31.403834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:31.404488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.404796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.405611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] PATH_INFO: `/v3/users/b8f7f53e4f3645f28540cd7b8f78938d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.408369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8f7f53e4f3645f28540cd7b8f78938d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:31.409736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:31.409736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 22 14:58:31.409736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: Sep 22 14:58:31.410980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:31.471259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c72f3-2f85-4916-b936-8e4f2be3aace admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:31.488584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:31.489209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.489752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.490639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] PATH_INFO: `/v3/projects/acb21b6887f24246ac0341d7882776cd` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.492469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb21b6887f24246ac0341d7882776cd)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:31.495532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:31.544803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8f7f53e4f3645f28540cd7b8f78938d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.558607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8f7f53e4f3645f28540cd7b8f78938d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.559549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8f7f53e4f3645f28540cd7b8f78938d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.573952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb21b6887f24246ac0341d7882776cd', 'request_id': 'req-079e2b58-4d83-4f5d-8397-524e25da79e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.585411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8f7f53e4f3645f28540cd7b8f78938d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.586274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb21b6887f24246ac0341d7882776cd', 'request_id': 'req-079e2b58-4d83-4f5d-8397-524e25da79e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.599390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5d6ad08-323b-40c5-90c9-25a82fc28eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8f7f53e4f3645f28540cd7b8f78938d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.605439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb21b6887f24246ac0341d7882776cd', 'request_id': 'req-079e2b58-4d83-4f5d-8397-524e25da79e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.610889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 841/1705] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:31 2022] DELETE /identity/v3/users/b8f7f53e4f3645f28540cd7b8f78938d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:31.616049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.617628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.618684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.626199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] The token cache is being invalidate because project acb21b6887f24246ac0341d7882776cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:31.626323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.628710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-079e2b58-4d83-4f5d-8397-524e25da79e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.632846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 865/1706] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:31 2022] DELETE /identity/v3/projects/acb21b6887f24246ac0341d7882776cd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:31.639832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b2e5-96b0-4a86-a797-5193c29dbf09 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.640138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b2e5-96b0-4a86-a797-5193c29dbf09 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.640621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b2e5-96b0-4a86-a797-5193c29dbf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.652742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:31.654077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:31.679504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:31.709226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4df9b2e5-96b0-4a86-a797-5193c29dbf09 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:31.713686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4df9b2e5-96b0-4a86-a797-5193c29dbf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:31.755501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 842/1707] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:31.766121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:31.770577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:31.856876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:31.882954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b8c9f-c396-474b-8c99-fd4cfe1c62ed None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:31.887765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:31.888403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.888683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.889060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.890145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 866/1708] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:31.898626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.899311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.899769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.915983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:31.917448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:31.925498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:31.926904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:31.950887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 867/1709] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:31.953739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:31.962122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 843/1710] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:31.966417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b95a0-0d9c-40d2-bad9-98585817d23e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:31.972284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:31.973111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.973546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.973964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.976099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:31.976665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:31.979618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.007474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 868/1711] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:32.008567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.021045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31107c14-3f4f-4df8-a203-016f2bf1def4 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.027313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.028339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.028630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.028906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.029571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.032251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.090821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 844/1712] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:32.097122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.109813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd95f0-2d0c-4099-b700-03ce1bd1b0ae admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.238250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.240104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.240675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.241150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.241686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.243846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.282026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 869/1713] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:32.286526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.313639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605729d8-ecc6-4f06-b51f-3f72c6f8fd7f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.339045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.339819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.340169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.340519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.341235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.343593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.370864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.371814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 845/1714] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:32.384478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dff3d-275c-46d5-bd02-14ef83d904c0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.390903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.391291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.391517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.391789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.392437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.394258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.443846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 870/1715] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:32.446388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.466576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552e894-0344-4a49-9d9e-6528c958e9d0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.474653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972366ac-76c9-4ab6-aa53-3cbcc8be856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.475468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972366ac-76c9-4ab6-aa53-3cbcc8be856b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.475764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972366ac-76c9-4ab6-aa53-3cbcc8be856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.476072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972366ac-76c9-4ab6-aa53-3cbcc8be856b admin admin] PATH_INFO: `/v3/projects/c13b2a205af040488a364d26eae23a15/users/bece5c63200640438821a7ef86709ed5/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.503535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.505496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.510077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972366ac-76c9-4ab6-aa53-3cbcc8be856b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13b2a205af040488a364d26eae23a15, user_id=bece5c63200640438821a7ef86709ed5, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.515880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972366ac-76c9-4ab6-aa53-3cbcc8be856b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.527458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bff7db-31a6-4d66-9da3-e6e0a21e81f8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.534576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.536417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.536761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.537496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] PATH_INFO: `/v3/projects/8fd1dbe224fe4121aad305172e9132a3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.539658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd1dbe224fe4121aad305172e9132a3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.542924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.552216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 846/1716] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:32 2022] PUT /identity/v3/projects/c13b2a205af040488a364d26eae23a15/users/bece5c63200640438821a7ef86709ed5/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:32.565793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.566074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.566362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.606421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] MFA Rules not processed for user `bece5c63200640438821a7ef86709ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:32.607757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.615154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd1dbe224fe4121aad305172e9132a3', 'request_id': 'req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.624871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd1dbe224fe4121aad305172e9132a3', 'request_id': 'req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.631576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd1dbe224fe4121aad305172e9132a3', 'request_id': 'req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.649835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] The token cache is being invalidate because project 8fd1dbe224fe4121aad305172e9132a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:32.650213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.651155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c4d0aed5-b6e5-495a-b616-e1dfe6dc73f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.655608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 871/1717] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:32 2022] DELETE /identity/v3/projects/8fd1dbe224fe4121aad305172e9132a3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:32.657983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332026-d22d-447f-a872-60668e4c026d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.658243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332026-d22d-447f-a872-60668e4c026d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.658590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83332026-d22d-447f-a872-60668e4c026d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.676940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:32.681561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 847/1718] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:32.682537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-83332026-d22d-447f-a872-60668e4c026d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:32.683951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-83332026-d22d-447f-a872-60668e4c026d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.685613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.688563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.728421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:32.753043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 872/1719] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:32.761064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.761273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.761546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.768063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a673c6-cb3f-4af7-8f86-ff4ee39665ae None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.772648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.773264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.773542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.773812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.774444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.776387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.781046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:32.782231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.790858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 873/1720] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:32.794618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.806128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4bd0a-677a-47ec-8637-a8af27d393da None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.807320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 848/1721] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:32.812476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.896864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.899319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c24164-f14a-4aaa-ad82-f9dc77b1f01d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.900704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.901079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.901355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.902033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:32.902342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.904634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.905912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.906709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.907892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.907892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.908235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.910092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.932650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 874/1722] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:58:32.937489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.957411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae93af-c38e-4d30-9543-c65cc54497d6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.966875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 849/1723] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:32.974429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.975415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.975415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.975680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.976315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.978258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.981123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.008500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef1e74-cb32-474e-ba87-bf18357c60a8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.023342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.023970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.024240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.024514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.025169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.026762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.042881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 850/1724] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:33.056747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.059299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 875/1725] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:32 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:33.072028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.083396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b554b5-395f-41db-8288-913116315ab9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.143492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5531-ff43-4c94-96ff-ab919f258bde admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.148088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.148697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.148891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.149017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.149236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.149556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.149839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.150001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.150098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] PATH_INFO: `/v3/projects/4fbe3294421f406da60509a89490c47e/users/735925da3d194bc4bc7fa0c7b8c1bd5a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.151980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.154056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:33.170476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fbe3294421f406da60509a89490c47e, user_id=735925da3d194bc4bc7fa0c7b8c1bd5a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.175062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.202563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 876/1726] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:33 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:33.207564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 851/1727] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:33 2022] PUT /identity/v3/projects/4fbe3294421f406da60509a89490c47e/users/735925da3d194bc4bc7fa0c7b8c1bd5a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:33.207757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.209127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.221767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0cbd3-59b0-4531-bc40-e9ba582b3cd0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.260683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:33.281657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c678b8-eb7f-4ecb-a32e-a8d808bbb232 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.286126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.286495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.286719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.287238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.287326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.287405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.287478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.287692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.287931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.288292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.289526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.289887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.303729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 852/1728] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:33.303920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 877/1729] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:33.305122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.305264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.315078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33905b35-f9f2-4e6a-80a3-9e124bc093d1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.315930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669ea1-394e-4d0b-a362-b238ffc300a5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.319772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907ec6d-12ed-4330-9f02-cd140cf899ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.320371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907ec6d-12ed-4330-9f02-cd140cf899ad admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.320471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.320653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907ec6d-12ed-4330-9f02-cd140cf899ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.320934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907ec6d-12ed-4330-9f02-cd140cf899ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.321139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.321389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.321668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] PATH_INFO: `/v3/projects/4fbe3294421f406da60509a89490c47e/users/735925da3d194bc4bc7fa0c7b8c1bd5a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.323947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fbe3294421f406da60509a89490c47e, user_id=735925da3d194bc4bc7fa0c7b8c1bd5a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.324494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907ec6d-12ed-4330-9f02-cd140cf899ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.326193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907ec6d-12ed-4330-9f02-cd140cf899ad admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.326845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.339145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 853/1730] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:33 2022] PUT /identity/v3/projects/4fbe3294421f406da60509a89490c47e/users/735925da3d194bc4bc7fa0c7b8c1bd5a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:33.339445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 878/1731] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:33.350152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.350887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.351133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.351471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.382727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] MFA Rules not processed for user `735925da3d194bc4bc7fa0c7b8c1bd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:33.383960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.421454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1687376-6429-49fb-83dd-9907a911a2d9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.426334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50593205-9007-4513-af22-611c0a5d165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.426994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50593205-9007-4513-af22-611c0a5d165e admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.427302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50593205-9007-4513-af22-611c0a5d165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.427902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50593205-9007-4513-af22-611c0a5d165e admin admin] PATH_INFO: `/v3/projects/c4fbdc61c8904b2daf1da7a43f29c3bf/users/694277390eed4e5dad1e6869727099e4/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.442508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50593205-9007-4513-af22-611c0a5d165e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4fbdc61c8904b2daf1da7a43f29c3bf, user_id=694277390eed4e5dad1e6869727099e4, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.445257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50593205-9007-4513-af22-611c0a5d165e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.459837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 854/1732] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:33 2022] PUT /identity/v3/projects/c4fbdc61c8904b2daf1da7a43f29c3bf/users/694277390eed4e5dad1e6869727099e4/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:33.471803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df718191-b806-4c60-b697-da5caffb946c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.472275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df718191-b806-4c60-b697-da5caffb946c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.472551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df718191-b806-4c60-b697-da5caffb946c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.489263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 879/1733] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:33.500613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.502398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.507726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-df718191-b806-4c60-b697-da5caffb946c None None] MFA Rules not processed for user `694277390eed4e5dad1e6869727099e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:33.508952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df718191-b806-4c60-b697-da5caffb946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.560387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:33.582603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648b43-d95d-4a8f-b2a0-94c1b9ed136d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.609045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 855/1734] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:33.623066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df718191-b806-4c60-b697-da5caffb946c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.627545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df718191-b806-4c60-b697-da5caffb946c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.667559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.668385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.669381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.669672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.700482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:33.807253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df718191-b806-4c60-b697-da5caffb946c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.809344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.815547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.825762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 880/1735] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:33.829182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966f61f-b8e6-4b68-87b4-122b4917db46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.830626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966f61f-b8e6-4b68-87b4-122b4917db46 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.830916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966f61f-b8e6-4b68-87b4-122b4917db46 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.831450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966f61f-b8e6-4b68-87b4-122b4917db46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.847086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966f61f-b8e6-4b68-87b4-122b4917db46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.848746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966f61f-b8e6-4b68-87b4-122b4917db46 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.860293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 856/1736] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:35.760902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.768006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:35.900819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bb58-77c2-4275-a710-cc3fdc70eea7 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.907095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.907871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.908174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.908449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] PATH_INFO: `/v3/users/0b4e842eed494513a2d1c3f86d6c3030` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.909796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b4e842eed494513a2d1c3f86d6c3030)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.915458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.993233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b4e842eed494513a2d1c3f86d6c3030'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.008283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b4e842eed494513a2d1c3f86d6c3030'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.031255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b4e842eed494513a2d1c3f86d6c3030'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.032196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b4e842eed494513a2d1c3f86d6c3030'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.033066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b4e842eed494513a2d1c3f86d6c3030'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.065788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 881/1737] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:35 2022] DELETE /identity/v3/users/0b4e842eed494513a2d1c3f86d6c3030 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:36.070784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.071357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.071861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.094379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:36.095658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:36.175272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 857/1738] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:36.185725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:36.187802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:36.234084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:36.260611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2fa7d-bf12-4881-ba20-4b5cf2650300 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:36.267000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:36.269158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.269432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.269711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.282434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:36.283927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:36.294192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 882/1739] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:36.602382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:36.603881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:36.617057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91adaa3-bf13-4060-93cf-3ace00a96b6b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:36.622064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:36.622739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.623019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.623340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] PATH_INFO: `/v3/projects/35d51e9cc4d343d487cf45c2051c11f2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.625018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d51e9cc4d343d487cf45c2051c11f2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:36.627133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:36.697433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d51e9cc4d343d487cf45c2051c11f2', 'request_id': 'req-64a8537d-7e7e-429f-9720-30b61f09c174'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.705456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d51e9cc4d343d487cf45c2051c11f2', 'request_id': 'req-64a8537d-7e7e-429f-9720-30b61f09c174'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.710616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:36.713058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d51e9cc4d343d487cf45c2051c11f2', 'request_id': 'req-64a8537d-7e7e-429f-9720-30b61f09c174'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.730751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] The token cache is being invalidate because project 35d51e9cc4d343d487cf45c2051c11f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:36.731825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.733860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.741923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 858/1740] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:36 2022] DELETE /identity/v3/projects/35d51e9cc4d343d487cf45c2051c11f2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:36.742769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:36.754737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281b2b-4098-496e-bc4a-bded02271da5 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:36.868590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:36.869695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.870072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.870351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] PATH_INFO: `/v3/users/6e958ad9a21b4cc884dcd33295037193` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.871737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e958ad9a21b4cc884dcd33295037193)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:36.876314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.003338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e958ad9a21b4cc884dcd33295037193'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.016894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e958ad9a21b4cc884dcd33295037193'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.042070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e958ad9a21b4cc884dcd33295037193'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.042842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e958ad9a21b4cc884dcd33295037193'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.043584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ce0ca750-cc59-4ec4-b05a-a952363e76cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e958ad9a21b4cc884dcd33295037193'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.065447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 883/1741] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:36 2022] DELETE /identity/v3/users/6e958ad9a21b4cc884dcd33295037193 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:37.462024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.463729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:37.478485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8537d-7e7e-429f-9720-30b61f09c174 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.550914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.551609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.551945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.552220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] PATH_INFO: `/v3/projects/63f87bdff23040d0b295d34cbaf800ba` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.553496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f87bdff23040d0b295d34cbaf800ba)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.555554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.614990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f87bdff23040d0b295d34cbaf800ba', 'request_id': 'req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.622415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f87bdff23040d0b295d34cbaf800ba', 'request_id': 'req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.628630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f87bdff23040d0b295d34cbaf800ba', 'request_id': 'req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.640593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] The token cache is being invalidate because project 63f87bdff23040d0b295d34cbaf800ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:37.640877 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.641512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a144d559-436a-4fd8-bcd5-0bbaafb8b92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.643018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 859/1742] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:37 2022] DELETE /identity/v3/projects/63f87bdff23040d0b295d34cbaf800ba => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:37.647921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632d638-0b54-482b-a079-124660d321c9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.648289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632d638-0b54-482b-a079-124660d321c9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.648494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632d638-0b54-482b-a079-124660d321c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.668639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3632d638-0b54-482b-a079-124660d321c9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:37.669824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3632d638-0b54-482b-a079-124660d321c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:37.719446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:37.732511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 884/1743] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:37.736474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.736761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.737035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.756684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:37.757876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:37.766443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 860/1744] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:37.770692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632d638-0b54-482b-a079-124660d321c9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.780234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632d638-0b54-482b-a079-124660d321c9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.784815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.785417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.785694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.785964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.786602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:37.786903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.788539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.799974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 885/1745] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:58:37.804237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.814055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd16a2-85ee-41e7-9e7e-d03a3a1694d3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.818632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.819251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.819539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.819812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.820451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.822346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.852102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 861/1746] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:37 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:37.856283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.866493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6084a4-ea17-4fa8-a32a-c873dc3edc0a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.926308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.926986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.927178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.927482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.928120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.930045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.950563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 886/1747] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:37 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:37.955416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.965611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224b780-eaeb-4be7-8769-0410cbf25621 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.970238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.970777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.971057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.971374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.972013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.973622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.983913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 862/1748] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:37.987851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.998082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d956f-5e5f-4760-9e38-9f07656de1c9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.003149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485bac4-b9ce-4ab8-a822-9d632ae1b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.003870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485bac4-b9ce-4ab8-a822-9d632ae1b90f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.004040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485bac4-b9ce-4ab8-a822-9d632ae1b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.004309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485bac4-b9ce-4ab8-a822-9d632ae1b90f admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.005063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485bac4-b9ce-4ab8-a822-9d632ae1b90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.006642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485bac4-b9ce-4ab8-a822-9d632ae1b90f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.018693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 887/1749] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:38.022781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.033338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036863d1-fb22-4fd1-88c9-cad3621a21d0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.038259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.038918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.039310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.039510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] PATH_INFO: `/v3/projects/a06baa9b0b0f46819fdaca1b90d25c48/users/fad21500d8c34fe5bd1b3d69b3271a41/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.055881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06baa9b0b0f46819fdaca1b90d25c48, user_id=fad21500d8c34fe5bd1b3d69b3271a41, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.058564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.072860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 863/1750] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:38 2022] PUT /identity/v3/projects/a06baa9b0b0f46819fdaca1b90d25c48/users/fad21500d8c34fe5bd1b3d69b3271a41/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:38.082991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.083315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.083597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.111474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] MFA Rules not processed for user `fad21500d8c34fe5bd1b3d69b3271a41`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:38.112678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:38.179539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 888/1751] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:38.184049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.203882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:38.261201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d0e2a-3730-4fee-9b51-5d92e644399f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.267176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.268192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.268509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.268799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.269443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.271401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.305066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 864/1752] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:38 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:38.306853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.320894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce529-3dcb-454c-8e4a-20741a09991e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.376368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:38.384139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.384784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.385603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.385877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.386523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.388506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.414472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 889/1753] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:38 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:38.415913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.427083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48253a-47b7-4bd2-8f2c-488226225ca1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.431704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.432310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.432589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.432862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.433593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.435311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.447762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 865/1754] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:38.451895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.462550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e03bf-1f5a-4d8b-a5fe-73e14d10096b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.468352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.469003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.469278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.469551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] PATH_INFO: `/v3/projects/ab54a46e1b1142219b4cc5dda7950690/users/dac88811445c47b79c26255632dfecc2/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.486797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab54a46e1b1142219b4cc5dda7950690, user_id=dac88811445c47b79c26255632dfecc2, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.489430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.508074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 890/1755] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:38 2022] PUT /identity/v3/projects/ab54a46e1b1142219b4cc5dda7950690/users/dac88811445c47b79c26255632dfecc2/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:38.509493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.521010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666a7c7-2131-45b6-a586-76f6f2b24482 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.590363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde6bb1-6336-4809-b87d-5dc7e0e09800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.590998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6bb1-6336-4809-b87d-5dc7e0e09800 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.591271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6bb1-6336-4809-b87d-5dc7e0e09800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.591548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6bb1-6336-4809-b87d-5dc7e0e09800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.592788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde6bb1-6336-4809-b87d-5dc7e0e09800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.594447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde6bb1-6336-4809-b87d-5dc7e0e09800 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.604953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 866/1756] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:38.610208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.623663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1a2dd-ee6c-49fd-8ddb-dd27ddf9db04 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.629485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.630448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.630738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.631016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] PATH_INFO: `/v3/projects/ab54a46e1b1142219b4cc5dda7950690/users/dac88811445c47b79c26255632dfecc2/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.633872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab54a46e1b1142219b4cc5dda7950690, user_id=dac88811445c47b79c26255632dfecc2, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.636958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.650227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 891/1757] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:38 2022] PUT /identity/v3/projects/ab54a46e1b1142219b4cc5dda7950690/users/dac88811445c47b79c26255632dfecc2/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:38.662327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.662624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.662905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.699691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] MFA Rules not processed for user `dac88811445c47b79c26255632dfecc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:38.701260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:38.793594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 867/1758] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:38.805631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.807596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:38.867846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e965618-d188-48b7-aa29-cb08bd7bfffc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.871476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d552134-ff80-42a9-aaec-6fd58826dbe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.872084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d552134-ff80-42a9-aaec-6fd58826dbe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.872365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d552134-ff80-42a9-aaec-6fd58826dbe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.872639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d552134-ff80-42a9-aaec-6fd58826dbe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.881573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d552134-ff80-42a9-aaec-6fd58826dbe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.882997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d552134-ff80-42a9-aaec-6fd58826dbe3 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.893333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 892/1759] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:38.925540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:38.937849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:38.990104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6b97e-1219-462e-bcdf-0d1b9257a1f0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:38.995072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:38.995735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:38.996018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:38.996289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] PATH_INFO: `/v3/users/fad21500d8c34fe5bd1b3d69b3271a41` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:38.997445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad21500d8c34fe5bd1b3d69b3271a41)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:38.999403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:39.046763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad21500d8c34fe5bd1b3d69b3271a41'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.054874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad21500d8c34fe5bd1b3d69b3271a41'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.055705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad21500d8c34fe5bd1b3d69b3271a41'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.067977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad21500d8c34fe5bd1b3d69b3271a41'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.074922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad21500d8c34fe5bd1b3d69b3271a41'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.081580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 868/1760] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:38 2022] DELETE /identity/v3/users/fad21500d8c34fe5bd1b3d69b3271a41 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:39.086143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.086645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.086753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.106038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:39.107268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:39.208408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 893/1761] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:39.232351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:39.233770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:39.310204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa06da2-6304-43cb-b9ab-c40761ad24ec admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:39.313950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:39.314558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.314830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.315115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.324981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:39.329585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:39.343095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 869/1762] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:39.693842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:39.697980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:39.731735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdefcd-da91-480d-a2d5-11e50deb8782 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:39.742249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:39.743035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.743648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.744017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] PATH_INFO: `/v3/projects/a06baa9b0b0f46819fdaca1b90d25c48` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.745636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06baa9b0b0f46819fdaca1b90d25c48)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:39.749401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:39.809021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:39.812917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06baa9b0b0f46819fdaca1b90d25c48', 'request_id': 'req-52c77ecc-e210-4293-b159-c3da7c974626'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.822474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06baa9b0b0f46819fdaca1b90d25c48', 'request_id': 'req-52c77ecc-e210-4293-b159-c3da7c974626'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.829154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06baa9b0b0f46819fdaca1b90d25c48', 'request_id': 'req-52c77ecc-e210-4293-b159-c3da7c974626'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.844214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] The token cache is being invalidate because project a06baa9b0b0f46819fdaca1b90d25c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:39.844506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.845109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.847140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 894/1763] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:39 2022] DELETE /identity/v3/projects/a06baa9b0b0f46819fdaca1b90d25c48 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:39.851246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:39.863600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff273f1-de58-4849-b02a-7506047d9968 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:39.899802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:39.932621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:39.933261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.933549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.933825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] PATH_INFO: `/v3/users/dac88811445c47b79c26255632dfecc2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.936637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac88811445c47b79c26255632dfecc2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:39.938648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.011137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac88811445c47b79c26255632dfecc2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.019978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac88811445c47b79c26255632dfecc2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.020678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac88811445c47b79c26255632dfecc2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.033585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac88811445c47b79c26255632dfecc2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.040692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-faf2eca4-88e7-40de-9260-c920043fcb4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac88811445c47b79c26255632dfecc2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.048082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 870/1764] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:39 2022] DELETE /identity/v3/users/dac88811445c47b79c26255632dfecc2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:40.380722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.382248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.413296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c77ecc-e210-4293-b159-c3da7c974626 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.483129 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.484064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.484345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.484621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] PATH_INFO: `/v3/projects/ab54a46e1b1142219b4cc5dda7950690` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.485780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab54a46e1b1142219b4cc5dda7950690)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.487747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.539439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab54a46e1b1142219b4cc5dda7950690', 'request_id': 'req-508e7f90-3551-4755-9d43-920aa9c37b67'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.546807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab54a46e1b1142219b4cc5dda7950690', 'request_id': 'req-508e7f90-3551-4755-9d43-920aa9c37b67'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.552306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab54a46e1b1142219b4cc5dda7950690', 'request_id': 'req-508e7f90-3551-4755-9d43-920aa9c37b67'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.563078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] The token cache is being invalidate because project ab54a46e1b1142219b4cc5dda7950690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:40.563395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.564022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-508e7f90-3551-4755-9d43-920aa9c37b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.565510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 895/1765] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:40 2022] DELETE /identity/v3/projects/ab54a46e1b1142219b4cc5dda7950690 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:40.570368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.570687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.570992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.590705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:40.592573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.652331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 871/1766] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:40.661003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.661294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.661573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.683799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:40.685046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.693663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 896/1767] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:40.697911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.707970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7462f-afbd-4fcd-926f-8630a89b0608 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.718973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.719875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.720364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.720780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.721872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:40.722323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.724550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.740162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 872/1768] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:40.744534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.754992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c2010-9dc4-44c5-a041-62aeb415bb3e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.759888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c0807-8634-409a-995a-c39081300194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.760757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c0807-8634-409a-995a-c39081300194 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.761038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c0807-8634-409a-995a-c39081300194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.761307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c0807-8634-409a-995a-c39081300194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.761941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c0807-8634-409a-995a-c39081300194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.763971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c0807-8634-409a-995a-c39081300194 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.809016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 897/1769] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:40.813219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.824926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eea84-8003-4c8c-8442-7521b84ea0aa admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.896487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:40.917001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.917525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.917804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.918076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.918719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.921417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.943900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 873/1770] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:40.948236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c0807-8634-409a-995a-c39081300194 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.958915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c0807-8634-409a-995a-c39081300194 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.963546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.964148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.964427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.964702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.965345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.968625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.981235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 898/1771] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:40.986838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.015454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a3045-610b-4b13-b738-4b86b311d7f9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.015454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3708601-3d26-4fe4-a9cf-307ee61526a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.015454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708601-3d26-4fe4-a9cf-307ee61526a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.015454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708601-3d26-4fe4-a9cf-307ee61526a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.015741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3708601-3d26-4fe4-a9cf-307ee61526a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.015741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3708601-3d26-4fe4-a9cf-307ee61526a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.015873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3708601-3d26-4fe4-a9cf-307ee61526a4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.027559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 874/1772] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:41.034522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.044785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258167-5fe4-427c-a053-64e7e824082c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.049737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.050396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.050671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.050945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] PATH_INFO: `/v3/projects/fd5ebd2ac1c14a10b1c5dc18cf4713de/users/bba47a45d8754f808ad9b313ad335d1d/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.067480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5ebd2ac1c14a10b1c5dc18cf4713de, user_id=bba47a45d8754f808ad9b313ad335d1d, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.070389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.083372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:41.098392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 899/1773] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:41 2022] PUT /identity/v3/projects/fd5ebd2ac1c14a10b1c5dc18cf4713de/users/bba47a45d8754f808ad9b313ad335d1d/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:41.100560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.100789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.101065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.132635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] MFA Rules not processed for user `bba47a45d8754f808ad9b313ad335d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:41.133899 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:41.211797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 875/1774] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:41.216502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.290884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d6023-f8b2-4165-9e99-f4357a8ea5a0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.296319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.296950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.297232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.297506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.298151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.300204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.330502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 900/1775] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:41.334866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.348235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46b245-98cf-4daa-97c7-723a79716950 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.419361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.420137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.420487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.420830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.421541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.423593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.459740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 876/1776] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:41.461674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.481495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea5e99-5d4a-4c2f-b4b4-0d2e3cc49d4e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.487333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.488042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.488393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.488806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.489610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.491793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.512227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 901/1777] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:41.517013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.530166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcfde2-5296-4207-88b5-781cc730d90a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.545897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.546754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.547097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.547748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] PATH_INFO: `/v3/projects/a2ec5079847e42128dcd9cbad6bf1124/users/a0bf1eaec6ef43869fff5ba4182b791b/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.583542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ec5079847e42128dcd9cbad6bf1124, user_id=a0bf1eaec6ef43869fff5ba4182b791b, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.586558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.605610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 877/1778] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:41 2022] PUT /identity/v3/projects/a2ec5079847e42128dcd9cbad6bf1124/users/a0bf1eaec6ef43869fff5ba4182b791b/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:41.609394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.619655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d8d59-03db-4bc8-9831-94c3b0d0baad admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.700244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac85ac5-2c9b-412d-9e07-3e951a775d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.701015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac85ac5-2c9b-412d-9e07-3e951a775d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.702076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac85ac5-2c9b-412d-9e07-3e951a775d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.702464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac85ac5-2c9b-412d-9e07-3e951a775d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.703537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac85ac5-2c9b-412d-9e07-3e951a775d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.706031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac85ac5-2c9b-412d-9e07-3e951a775d5e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.717313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 902/1779] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:41.722329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.734026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c076d24-f520-405b-b7ef-a962593506e2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.738779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.739530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.739808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.740077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] PATH_INFO: `/v3/projects/a2ec5079847e42128dcd9cbad6bf1124/users/a0bf1eaec6ef43869fff5ba4182b791b/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.742019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ec5079847e42128dcd9cbad6bf1124, user_id=a0bf1eaec6ef43869fff5ba4182b791b, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.744719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.753135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:41.759014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 878/1780] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:41 2022] PUT /identity/v3/projects/a2ec5079847e42128dcd9cbad6bf1124/users/a0bf1eaec6ef43869fff5ba4182b791b/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:41.771000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d922f-a680-4115-a594-0be252a999af None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.771304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d922f-a680-4115-a594-0be252a999af None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.771579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d922f-a680-4115-a594-0be252a999af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.810553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0c0d922f-a680-4115-a594-0be252a999af None None] MFA Rules not processed for user `a0bf1eaec6ef43869fff5ba4182b791b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:41.812439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0c0d922f-a680-4115-a594-0be252a999af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:41.887234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:41.902484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 903/1781] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:41.931766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.933628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:42.005943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551294c-7116-4b37-aae7-95cc4cb1222e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.009478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.010087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.010619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.010891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.021135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.022620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.033838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 879/1782] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:42.357029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d922f-a680-4115-a594-0be252a999af None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.367519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d922f-a680-4115-a594-0be252a999af None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.372237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0bf1eaec6ef43869fff5ba4182b791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2ec5079847e42128dcd9cbad6bf1124', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.372862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.373333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.373602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.374552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:42.374858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.376524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.389102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 904/1783] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:42.393344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.402448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766c5f-9b76-4944-9878-65c242a96238 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.407523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0bf1eaec6ef43869fff5ba4182b791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2ec5079847e42128dcd9cbad6bf1124', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.408113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.408393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.408666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.409296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.411183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.437871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 880/1784] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:42 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:42.442321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.453452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751baf51-a126-4552-933c-a0820b982b02 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.530789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0bf1eaec6ef43869fff5ba4182b791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2ec5079847e42128dcd9cbad6bf1124', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.531494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.531759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.532016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.532676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:42.532986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.534698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.548332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 905/1785] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:42.594604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.605536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00dd26-951c-48e0-b676-6d0ba0b38e9f tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.610248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0bf1eaec6ef43869fff5ba4182b791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a2ec5079847e42128dcd9cbad6bf1124', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.610874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.611162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.611476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] PATH_INFO: `/v3/projects/c8b07d0e6a664d9a955df6b2fae9d5db` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.612602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c8b07d0e6a664d9a955df6b2fae9d5db)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.614504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.671947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b07d0e6a664d9a955df6b2fae9d5db', 'request_id': 'req-88e6fbed-6446-4a44-b630-3e144aa93290'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:42.692587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b07d0e6a664d9a955df6b2fae9d5db', 'request_id': 'req-88e6fbed-6446-4a44-b630-3e144aa93290'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:42.701797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b07d0e6a664d9a955df6b2fae9d5db', 'request_id': 'req-88e6fbed-6446-4a44-b630-3e144aa93290'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:42.715279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] The token cache is being invalidate because project c8b07d0e6a664d9a955df6b2fae9d5db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:42.715563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:42.717084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88e6fbed-6446-4a44-b630-3e144aa93290 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:42.722832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 881/1786] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:42 2022] DELETE /identity/v3/projects/c8b07d0e6a664d9a955df6b2fae9d5db => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:42.937484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.939343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.019118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c2459-1c47-4a42-b6f1-3aad3d76f7b2 tempest-VolumeQuotaClassesTest-1242692999 tempest-VolumeQuotaClassesTest-1242692999-project-admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.089611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.090181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.090461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.090733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] PATH_INFO: `/v3/users/bba47a45d8754f808ad9b313ad335d1d` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.092401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba47a45d8754f808ad9b313ad335d1d)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.094347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.126820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:43.146776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba47a45d8754f808ad9b313ad335d1d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.164968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba47a45d8754f808ad9b313ad335d1d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.179877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba47a45d8754f808ad9b313ad335d1d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.180880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba47a45d8754f808ad9b313ad335d1d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.181679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba47a45d8754f808ad9b313ad335d1d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.194891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 906/1787] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:42 2022] DELETE /identity/v3/users/bba47a45d8754f808ad9b313ad335d1d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:43.199525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.200017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.200130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.220691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:43.222078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.286014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 882/1788] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:43.296513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:43.298073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.358733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8e552-4c17-4ec8-adc9-806f512277e6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.362316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.363265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.363265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.363501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.372809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.374334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.384975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 907/1789] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:43.697612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:43.699553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.719932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:43.737847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bcce8-1120-4106-be2f-5ab8d6999a49 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.742524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.743325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.743670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.744005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] PATH_INFO: `/v3/projects/fd5ebd2ac1c14a10b1c5dc18cf4713de` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.745695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5ebd2ac1c14a10b1c5dc18cf4713de)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.747647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.800539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5ebd2ac1c14a10b1c5dc18cf4713de', 'request_id': 'req-261d7ba3-4a88-454b-9d09-fd9e04a77394'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.809107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5ebd2ac1c14a10b1c5dc18cf4713de', 'request_id': 'req-261d7ba3-4a88-454b-9d09-fd9e04a77394'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.816202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5ebd2ac1c14a10b1c5dc18cf4713de', 'request_id': 'req-261d7ba3-4a88-454b-9d09-fd9e04a77394'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.828960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] The token cache is being invalidate because project fd5ebd2ac1c14a10b1c5dc18cf4713de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:43.829249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.830020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.831470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 883/1790] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:43 2022] DELETE /identity/v3/projects/fd5ebd2ac1c14a10b1c5dc18cf4713de => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:43.835458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:43.845178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c23389-b961-4282-87b5-c60298b89406 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.913999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.914640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.914948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.915305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] PATH_INFO: `/v3/users/a0bf1eaec6ef43869fff5ba4182b791b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.916446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bf1eaec6ef43869fff5ba4182b791b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.918414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.967768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bf1eaec6ef43869fff5ba4182b791b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.969519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:43.977158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bf1eaec6ef43869fff5ba4182b791b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.991420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bf1eaec6ef43869fff5ba4182b791b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.992119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bf1eaec6ef43869fff5ba4182b791b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:43.992729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bf1eaec6ef43869fff5ba4182b791b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.010012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 908/1791] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:43 2022] DELETE /identity/v3/users/a0bf1eaec6ef43869fff5ba4182b791b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:44.149354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.151409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.248439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7ba3-4a88-454b-9d09-fd9e04a77394 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.304504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.305135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.305394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.306865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.308378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.401915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:44.411080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.414665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87421d3-afd1-474b-bfbb-7c6533cfa623 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.429427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 884/1792] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:44.501386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.503176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.593935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88131ee1-8b08-4e58-8e32-8be56a1f47b2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.602110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.602753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.603031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.603465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] PATH_INFO: `/v3/projects/a2ec5079847e42128dcd9cbad6bf1124` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.604609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ec5079847e42128dcd9cbad6bf1124)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.606575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.661649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ec5079847e42128dcd9cbad6bf1124', 'request_id': 'req-c12e2b23-41cf-4cbb-a156-00c74ed8b898'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.669586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ec5079847e42128dcd9cbad6bf1124', 'request_id': 'req-c12e2b23-41cf-4cbb-a156-00c74ed8b898'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.675772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ec5079847e42128dcd9cbad6bf1124', 'request_id': 'req-c12e2b23-41cf-4cbb-a156-00c74ed8b898'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.690781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] The token cache is being invalidate because project a2ec5079847e42128dcd9cbad6bf1124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:44.691074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.692113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c12e2b23-41cf-4cbb-a156-00c74ed8b898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.693850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 909/1793] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:44 2022] DELETE /identity/v3/projects/a2ec5079847e42128dcd9cbad6bf1124 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:44.699246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.699538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.699822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.723096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:44.724410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.808977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 885/1794] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:44.817872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.818178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.820389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.828306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.828306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.849330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76103-c4bc-4bc4-aaa9-dbbfb19e107c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.854649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:44.855940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.859267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.860570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.860950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.861343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] PATH_INFO: `/v3/users/694277390eed4e5dad1e6869727099e4` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.863006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=694277390eed4e5dad1e6869727099e4)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.865623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.866719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 910/1795] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:44.871001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.881996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a057-11ae-4b75-a4be-dbd2a6ab560b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.887265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.887948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.888149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.888527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.889280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:44.889600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.892774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.903790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 911/1796] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:58:44.908263 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.919191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b40f-1db9-468b-b587-49eab7938c2f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.925479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.926024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.926411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.926593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.927319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.929636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.930589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '694277390eed4e5dad1e6869727099e4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.939711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '694277390eed4e5dad1e6869727099e4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.943284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '694277390eed4e5dad1e6869727099e4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.972532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '694277390eed4e5dad1e6869727099e4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.975102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 912/1797] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:44.978556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.981007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3c6847e1-7764-4e54-8b0f-943a66a08e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '694277390eed4e5dad1e6869727099e4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.992225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 886/1798] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:44 2022] DELETE /identity/v3/users/694277390eed4e5dad1e6869727099e4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:44.993920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e820-1d56-4991-9e27-774395426b50 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.994193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e820-1d56-4991-9e27-774395426b50 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.994473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629e820-1d56-4991-9e27-774395426b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.005803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.018331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b629e820-1d56-4991-9e27-774395426b50 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:45.019914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b629e820-1d56-4991-9e27-774395426b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.084468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5704-5b9d-4538-8fbf-2d0cfc11073f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.109487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 887/1799] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:45.117885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629e820-1d56-4991-9e27-774395426b50 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.119841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b629e820-1d56-4991-9e27-774395426b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.132971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.162195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.163244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.163729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.164163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.165155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.168794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.215514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629e820-1d56-4991-9e27-774395426b50 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.215598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 913/1800] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:45.216190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.219092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.220101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.220765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.221053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.221320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.232007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.234204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.234337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aae357-9a7d-4898-89d6-10493660003c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.242582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.243612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.243904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.244239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.244900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.246195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 888/1801] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:45.246746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.261811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 914/1802] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:45.266588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.280306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaefa58-b5b4-4b32-ab40-46a84765407b service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.288287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968a704-f66d-44b4-b466-8aa680ae4446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.289105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968a704-f66d-44b4-b466-8aa680ae4446 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.289408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968a704-f66d-44b4-b466-8aa680ae4446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.289681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968a704-f66d-44b4-b466-8aa680ae4446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.290501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968a704-f66d-44b4-b466-8aa680ae4446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.292071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968a704-f66d-44b4-b466-8aa680ae4446 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.307247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 889/1803] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:45.309664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.322517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530196d-1daa-4fef-b9fa-26f97daf5760 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.330529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.331310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.331559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.331847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] PATH_INFO: `/v3/projects/7c5f63cce75d4cfe875091dcf3957ed9/users/9ef2a995fe994166855c907a97dd78f7/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.354628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5f63cce75d4cfe875091dcf3957ed9, user_id=9ef2a995fe994166855c907a97dd78f7, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.358055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.376900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 915/1804] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:45 2022] PUT /identity/v3/projects/7c5f63cce75d4cfe875091dcf3957ed9/users/9ef2a995fe994166855c907a97dd78f7/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:45.386193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.386412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.386643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.422336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] MFA Rules not processed for user `9ef2a995fe994166855c907a97dd78f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:45.424648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.543315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 890/1805] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:45.550532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.604828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.608400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.652783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4f03-153d-4e91-9507-6a145513cbb0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.664109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.665664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.666233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.666768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.668405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.673131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.701336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.715409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 916/1806] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:45.716611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.719668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fdc8d-5c97-4b93-925f-916609363d87 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.767457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.782112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7705a6f-702d-4639-9d3a-83a5147a639b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.786718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.787358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.787628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.787889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.788513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.790434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.792919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.793555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.793837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.794142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] PATH_INFO: `/v3/projects/c4fbdc61c8904b2daf1da7a43f29c3bf` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.795832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4fbdc61c8904b2daf1da7a43f29c3bf)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.797741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.814669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 917/1807] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:45.818368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.828837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc0bb-5f7b-45bc-acdb-51401cfc6344 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.833969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.834568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.834853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.835189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.835798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.837418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.846642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4fbdc61c8904b2daf1da7a43f29c3bf', 'request_id': 'req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.848874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 918/1808] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:45.852998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.857204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4fbdc61c8904b2daf1da7a43f29c3bf', 'request_id': 'req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.864251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4fbdc61c8904b2daf1da7a43f29c3bf', 'request_id': 'req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.864939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98171a0-d2b3-4a9e-a143-f1d6467fa49b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.877028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] The token cache is being invalidate because project c4fbdc61c8904b2daf1da7a43f29c3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:45.877320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.878050 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3a4dbb93-9cbb-40fc-8ed5-a9d70c46424d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.879892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 891/1809] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:45 2022] DELETE /identity/v3/projects/c4fbdc61c8904b2daf1da7a43f29c3bf => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:45.884325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcf95f-63e9-4cd1-8991-8c9f5d531b7b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.885713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcf95f-63e9-4cd1-8991-8c9f5d531b7b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.885713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcf95f-63e9-4cd1-8991-8c9f5d531b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.913342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a3dcf95f-63e9-4cd1-8991-8c9f5d531b7b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:45.914619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a3dcf95f-63e9-4cd1-8991-8c9f5d531b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.934209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.934789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.935075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.935385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] PATH_INFO: `/v3/projects/e1084d880a04469a98cc40375aeef3cd/users/e20881d44a93429699ebe9aefe6029ed/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.952195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1084d880a04469a98cc40375aeef3cd, user_id=e20881d44a93429699ebe9aefe6029ed, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.955023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.972984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 919/1810] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:45 2022] PUT /identity/v3/projects/e1084d880a04469a98cc40375aeef3cd/users/e20881d44a93429699ebe9aefe6029ed/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:45.974575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.976325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.999088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 892/1811] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:46.001047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.001344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.001681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.021698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.022919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.045489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d43c46-65c7-47d8-8eb7-ea5e31a31e42 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.050944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.051618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.051922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.052170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.052942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.054555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.057160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.070904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 920/1812] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.071847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.082094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98175-6d44-4a36-9f38-f58c21681f57 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.087249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218de4b-eeec-423a-b978-059501a4b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.087919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218de4b-eeec-423a-b978-059501a4b39d admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.088199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218de4b-eeec-423a-b978-059501a4b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.088474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218de4b-eeec-423a-b978-059501a4b39d admin admin] PATH_INFO: `/v3/projects/e1084d880a04469a98cc40375aeef3cd/users/e20881d44a93429699ebe9aefe6029ed/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.089079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 893/1813] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:46.093272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218de4b-eeec-423a-b978-059501a4b39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1084d880a04469a98cc40375aeef3cd, user_id=e20881d44a93429699ebe9aefe6029ed, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.093579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.096214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218de4b-eeec-423a-b978-059501a4b39d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.103250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445810a1-793a-4e81-9f9a-9db55378828d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.108346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.111954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 921/1814] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:46 2022] PUT /identity/v3/projects/e1084d880a04469a98cc40375aeef3cd/users/e20881d44a93429699ebe9aefe6029ed/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:46.123499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.123787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.124089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.162347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] MFA Rules not processed for user `e20881d44a93429699ebe9aefe6029ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.163296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.180245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.180865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.181147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.181419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.182067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:46.182373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.184010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.196393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 894/1815] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:46.200860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.211645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6609121-0096-4089-bb23-ec135c177f16 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.217437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.218725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.219091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.219351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.219934 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.221850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.247732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 922/1816] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:46.252956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 895/1817] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:46.256156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.256945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.257563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.336081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df19563-8658-496f-8c9f-64c3d5e8add0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.351079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640f488-cb17-4f71-b648-604b6a8d5dc7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.356453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.357061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.357374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.357661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.358285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.360230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.386808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 896/1818] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/users => generated 282 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:46.391378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.391784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3da4e-7831-4f7f-8616-0322310ca8c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.392408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3da4e-7831-4f7f-8616-0322310ca8c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.392683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3da4e-7831-4f7f-8616-0322310ca8c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.392951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3da4e-7831-4f7f-8616-0322310ca8c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.401867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6897f-cad6-4c80-94d2-3ba29099e12b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.407530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.408124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.408404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.408670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.409302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.410896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.420887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 897/1819] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.425226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.434502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.434959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fb60d-f712-41df-bf19-f6002a41ae60 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.442083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.442630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.442921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.443217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.443880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.445489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.459379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 898/1820] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.461133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3da4e-7831-4f7f-8616-0322310ca8c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.464261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3da4e-7831-4f7f-8616-0322310ca8c7 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.464956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.481022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043e431-c2bf-4203-b67f-e1000767ae3e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.481885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 923/1821] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:46.486264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.487337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.487823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.488251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] PATH_INFO: `/v3/projects/2d6b268c039440c99ad2d9768fe13864/users/d261890b49e2493884f2f8b6b0f2f574/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.504533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6b268c039440c99ad2d9768fe13864, user_id=d261890b49e2493884f2f8b6b0f2f574, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.516120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.532925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 899/1822] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:46 2022] PUT /identity/v3/projects/2d6b268c039440c99ad2d9768fe13864/users/d261890b49e2493884f2f8b6b0f2f574/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:46.543138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910b18-9289-436b-8aad-339302735f1f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.543494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910b18-9289-436b-8aad-339302735f1f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.543769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910b18-9289-436b-8aad-339302735f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.572340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-80910b18-9289-436b-8aad-339302735f1f None None] MFA Rules not processed for user `d261890b49e2493884f2f8b6b0f2f574`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.573561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-80910b18-9289-436b-8aad-339302735f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.632964 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.653774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 924/1823] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:46.661324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80910b18-9289-436b-8aad-339302735f1f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.683650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.734139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadf69b-e864-4a91-87bb-3b27e75addc9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.736657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80910b18-9289-436b-8aad-339302735f1f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.741016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.741431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.741811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.742194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.742342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.742620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.742865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] PATH_INFO: `/v3/users/9ef2a995fe994166855c907a97dd78f7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.742973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.743704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.745200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef2a995fe994166855c907a97dd78f7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.745940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.749735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.791398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 925/1824] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:46.793136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.827831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef2a995fe994166855c907a97dd78f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:46.837434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef2a995fe994166855c907a97dd78f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:46.838468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef2a995fe994166855c907a97dd78f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:46.852896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef2a995fe994166855c907a97dd78f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:46.861796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-71009777-d22a-4817-98d7-e8fc523a5b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef2a995fe994166855c907a97dd78f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:46.863686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.869019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 900/1825] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:46 2022] DELETE /identity/v3/users/9ef2a995fe994166855c907a97dd78f7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:46.874236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.874531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.874807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.881130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1ffb9-99d9-443e-a213-1b4111460bb3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.899089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.899089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.955530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.956260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.956644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.956948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.957568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.959559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.987343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 926/1826] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:46.987536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 901/1827] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:46.992561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.994460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.999638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.001206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.016266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5144-2249-40ad-87ae-a0102e584fdc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.024199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.025022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.025304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.025591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.026286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.028009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.047246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 927/1828] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:47.048438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.068592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:47.073683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b978166-0528-4331-8bf3-ec9a32f434b1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.082159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.082854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.083166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.083476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] PATH_INFO: `/v3/projects/c90af326234145f1b71ec49e84946686/users/1d376e81d6e84ed4b0c6c681e1ab2e5c/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.092669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326146a-504e-46a1-8c9c-bfa4ceb26017 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.099893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.101895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.102812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.103349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.109238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90af326234145f1b71ec49e84946686, user_id=1d376e81d6e84ed4b0c6c681e1ab2e5c, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.113291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.122906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.125177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.145410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 928/1830] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:47 2022] PUT /identity/v3/projects/c90af326234145f1b71ec49e84946686/users/1d376e81d6e84ed4b0c6c681e1ab2e5c/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:47.148381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 902/1829] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:47.150119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.242202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d052d97-c34b-42b1-91b6-79e90b31243f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.247158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f992b-ea8b-4f06-83fe-d754ac1c3c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.247852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f992b-ea8b-4f06-83fe-d754ac1c3c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.248170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f992b-ea8b-4f06-83fe-d754ac1c3c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.249038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f992b-ea8b-4f06-83fe-d754ac1c3c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.249175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f992b-ea8b-4f06-83fe-d754ac1c3c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.251256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f992b-ea8b-4f06-83fe-d754ac1c3c9d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.262514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 929/1831] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:47.267272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.280721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75cd55-7379-4968-a744-7f2831ce632d service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.300554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.301539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.301909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.302299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] PATH_INFO: `/v3/projects/c90af326234145f1b71ec49e84946686/users/1d376e81d6e84ed4b0c6c681e1ab2e5c/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.304506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90af326234145f1b71ec49e84946686, user_id=1d376e81d6e84ed4b0c6c681e1ab2e5c, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.307408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.320134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 903/1832] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:47 2022] PUT /identity/v3/projects/c90af326234145f1b71ec49e84946686/users/1d376e81d6e84ed4b0c6c681e1ab2e5c/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:47.332132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.332590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.332697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.387331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] MFA Rules not processed for user `1d376e81d6e84ed4b0c6c681e1ab2e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:47.389657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.466290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:47.510894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 930/1833] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:47.524265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.527548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.623066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619c8e0-dea8-43c2-a6cb-c22a55a0ec49 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.626844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.627511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.627797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.628076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.638848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.640371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.656421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 904/1834] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:47.681650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.683501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.751410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.760592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519125d3-08f8-4b53-9f20-67d3635dab38 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.765127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.765842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.766183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.766540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.768567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.771389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dfa30-e476-4e62-a794-3d8d10a94575 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.777775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.778534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.778891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.779958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] PATH_INFO: `/v3/projects/7c5f63cce75d4cfe875091dcf3957ed9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.781382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5f63cce75d4cfe875091dcf3957ed9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.783449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.838880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5f63cce75d4cfe875091dcf3957ed9', 'request_id': 'req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:47.847850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5f63cce75d4cfe875091dcf3957ed9', 'request_id': 'req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:47.848051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.850888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.853916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5f63cce75d4cfe875091dcf3957ed9', 'request_id': 'req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:47.862283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 905/1835] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:47.869491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] The token cache is being invalidate because project 7c5f63cce75d4cfe875091dcf3957ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:47.869786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:47.870459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:47.875288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 931/1836] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:47 2022] DELETE /identity/v3/projects/7c5f63cce75d4cfe875091dcf3957ed9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:47.876596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.889686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ce05d-2638-472f-b82d-0430d08d39db service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.961328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.962228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.962509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.962778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] PATH_INFO: `/v3/users/e20881d44a93429699ebe9aefe6029ed` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.964056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20881d44a93429699ebe9aefe6029ed)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.965965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.981581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:48.023525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20881d44a93429699ebe9aefe6029ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.030835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20881d44a93429699ebe9aefe6029ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.031511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20881d44a93429699ebe9aefe6029ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.043487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20881d44a93429699ebe9aefe6029ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.050733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ec5039e1-8c70-4aa3-a4a6-9fffbe6207fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20881d44a93429699ebe9aefe6029ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.056641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 906/1837] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:47 2022] DELETE /identity/v3/users/e20881d44a93429699ebe9aefe6029ed => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:48.364290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.365718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:48.389237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26a1a4-71e0-4e6d-856a-7c0c4545e271 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.474843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.475863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.476419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.477281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] PATH_INFO: `/v3/projects/e1084d880a04469a98cc40375aeef3cd` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.478592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1084d880a04469a98cc40375aeef3cd)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:48.481008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:48.497008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:48.548739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1084d880a04469a98cc40375aeef3cd', 'request_id': 'req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.556368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1084d880a04469a98cc40375aeef3cd', 'request_id': 'req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.562331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1084d880a04469a98cc40375aeef3cd', 'request_id': 'req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.573463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] The token cache is being invalidate because project e1084d880a04469a98cc40375aeef3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:48.573749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.574386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-cc3b4f4f-83f9-4e84-98ff-04f7a85d597c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:48.575903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 932/1838] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:48 2022] DELETE /identity/v3/projects/e1084d880a04469a98cc40375aeef3cd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:48.582121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.582624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.582709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.605216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:48.606501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:48.677487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 907/1839] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:48.685385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.685665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.685944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.707012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:48.708408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:48.720332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 933/1840] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:48.724652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.734837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad589763-82e2-4cc7-9fab-364eed115b39 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.739385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.739757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.740033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.740310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.740963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:48.741271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:48.742881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:48.755190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 908/1841] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:48.759729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.769906 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2096c82-cae5-402a-ad13-ed494fd9cfd0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.774544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.775226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.775653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.776169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.776773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:48.778766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:48.807618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 934/1842] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:48 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:48.811884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.821765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e7281-7185-4288-a1b2-a2cf94403096 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.879501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.880115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.880396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.880668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.881314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:48.883316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:48.907810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 909/1843] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:48 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:48.911940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.922798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11397b8e-ec9f-42dc-970b-190b70c9c425 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.927465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.928130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.928348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.928620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.929266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:48.930871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:48.941809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 935/1844] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:48.945813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.956692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f5c3f-7f2e-4a80-bd94-16ff102e7760 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.961842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88961046-1fcb-49a3-937c-07fcf06cad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.962517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88961046-1fcb-49a3-937c-07fcf06cad0f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.962802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88961046-1fcb-49a3-937c-07fcf06cad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.963237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88961046-1fcb-49a3-937c-07fcf06cad0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:48.963908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88961046-1fcb-49a3-937c-07fcf06cad0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:48.965496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88961046-1fcb-49a3-937c-07fcf06cad0f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:48.967844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:48.987798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 910/1845] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:48.987894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:48.992321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367e03-247d-43ef-b12f-8bf1c5b22a02 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:48.997082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78732d-48ee-4e33-a3c6-e4c3f92bde68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:48.998293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78732d-48ee-4e33-a3c6-e4c3f92bde68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:48.998575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78732d-48ee-4e33-a3c6-e4c3f92bde68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:48.998846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78732d-48ee-4e33-a3c6-e4c3f92bde68 admin admin] PATH_INFO: `/v3/projects/e343f78042c0450794e592f1ea1abc4a/users/b5916793c840454a9aab5bfdd46fea9d/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.014431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78732d-48ee-4e33-a3c6-e4c3f92bde68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e343f78042c0450794e592f1ea1abc4a, user_id=b5916793c840454a9aab5bfdd46fea9d, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.017101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78732d-48ee-4e33-a3c6-e4c3f92bde68 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.030846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 936/1846] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:48 2022] PUT /identity/v3/projects/e343f78042c0450794e592f1ea1abc4a/users/b5916793c840454a9aab5bfdd46fea9d/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:49.042384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.042677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.043111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.061156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.061468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.061801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.089856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] MFA Rules not processed for user `60c809b5a47a4983be26116553934387`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:49.095406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:49.101246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] MFA Rules not processed for user `b5916793c840454a9aab5bfdd46fea9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:49.102493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:49.157058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 937/1847] 10.222.0.49 () {58 vars in 1015 bytes} [Thu Sep 22 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:49.236054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.237907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:49.265093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853991b8-786e-4c70-b1b5-5a908dfa331f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.265822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 911/1848] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:49.266937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.274288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.274936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.275231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.275517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.284360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.285816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.295465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 938/1849] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:49.296297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.315950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723bca7-6a4e-4e74-996e-887c29295523 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.335928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:49.349932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.350561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.350843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.351118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.385609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.387084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.390660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0e92-4062-4ac6-9e23-1df85239d41f service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.396566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.397650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.398075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.398634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.399463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.402045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.421991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 912/1850] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:49.461774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 939/1851] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:49 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:49.466063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.489198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f693f8-de28-4724-891c-a82871440256 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.610497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.611574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.611574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.611936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.612701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.614724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.643631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 913/1852] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:49 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:49.646336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.659277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af690d8d-ab70-4a93-96ae-6177ba286d89 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.665094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.665879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.666163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.666460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.667603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.670549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.683478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 940/1853] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:49.688034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.717938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec9809-a8e5-4a21-96cf-66542cc86fff admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.724200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3d8ce-0631-4a40-b494-f1f094a68601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.725078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3d8ce-0631-4a40-b494-f1f094a68601 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.725476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3d8ce-0631-4a40-b494-f1f094a68601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.725931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3d8ce-0631-4a40-b494-f1f094a68601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.727253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3d8ce-0631-4a40-b494-f1f094a68601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.729404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3d8ce-0631-4a40-b494-f1f094a68601 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.747422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 914/1854] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:49.752340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:49.774104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf4c8d-e3bd-432a-a009-83428136a21e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:49.781181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:49.783543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.783903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.784257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] PATH_INFO: `/v3/projects/16b90495aaf34a308d48291e664baa12/users/036542e3801c4f148ab45391ba57db2d/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.808592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b90495aaf34a308d48291e664baa12, user_id=036542e3801c4f148ab45391ba57db2d, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:49.813682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:49.848617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 941/1855] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:49 2022] PUT /identity/v3/projects/16b90495aaf34a308d48291e664baa12/users/036542e3801c4f148ab45391ba57db2d/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:49.850231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:49.850725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:49.851011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:49.902015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] MFA Rules not processed for user `036542e3801c4f148ab45391ba57db2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:49.904733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:49.986311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:50.079190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 915/1856] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:50.080978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.229553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efaa60-6ee8-4c8f-af60-715d2fe212a6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.240394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.241159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.242356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.243959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.244784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.257396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.309709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 942/1857] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:50 2022] POST /identity/v3/projects => generated 377 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:50.315033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.336659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b5077-b4b9-44bd-853e-2550419e6da5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.450173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.451638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.451925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.452197 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.453881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.455979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.485924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 916/1858] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:50 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:50.489724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.502662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70abcb-84d0-4783-9c2a-13713fb6ba34 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.508692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.509456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.509817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.510617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.511429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.513185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.530716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 943/1859] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:50.532533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.547977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0298b28-15ba-4393-bff0-58fcf4fb0e1a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.562438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.563126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.563444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.563719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] PATH_INFO: `/v3/projects/f99fffebf6874d8baefd12f3571c33a0/users/eba035a59725442eb5b39605e06d917d/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.602620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99fffebf6874d8baefd12f3571c33a0, user_id=eba035a59725442eb5b39605e06d917d, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.605750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.628705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 917/1860] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:50 2022] PUT /identity/v3/projects/f99fffebf6874d8baefd12f3571c33a0/users/eba035a59725442eb5b39605e06d917d/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:50.632777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.641876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.645440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:50.648896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c3a5f-ec97-422a-b624-fa29821c43da admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.760965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f742a6-7db5-42f1-bb6a-f51dedeb260b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.772614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.773291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.775003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.775003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] PATH_INFO: `/v3/users/e3440f8c4bc44b8f89070b876dec874d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.777800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3440f8c4bc44b8f89070b876dec874d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.781480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.784036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:50.813579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.814583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.815222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.815706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.816594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.820118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.847304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 944/1861] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:50.849938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:50.867867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3440f8c4bc44b8f89070b876dec874d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:50.870996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:50.877718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b14a30-c4c3-471c-b029-1656aee1c009 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:50.882347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3440f8c4bc44b8f89070b876dec874d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:50.884385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3440f8c4bc44b8f89070b876dec874d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:50.889880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adedc14-dbb4-4da8-8552-43cc739d8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:50.891579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adedc14-dbb4-4da8-8552-43cc739d8b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.892083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adedc14-dbb4-4da8-8552-43cc739d8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.892572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adedc14-dbb4-4da8-8552-43cc739d8b50 admin admin] PATH_INFO: `/v3/projects/f99fffebf6874d8baefd12f3571c33a0/users/eba035a59725442eb5b39605e06d917d/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.895001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adedc14-dbb4-4da8-8552-43cc739d8b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99fffebf6874d8baefd12f3571c33a0, user_id=eba035a59725442eb5b39605e06d917d, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:50.898047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adedc14-dbb4-4da8-8552-43cc739d8b50 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:50.900628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3440f8c4bc44b8f89070b876dec874d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:50.911292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0a670e7b-ee03-4a7a-9312-61a3264ac409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3440f8c4bc44b8f89070b876dec874d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:50.928593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 945/1862] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:50 2022] PUT /identity/v3/projects/f99fffebf6874d8baefd12f3571c33a0/users/eba035a59725442eb5b39605e06d917d/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:50.932016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 918/1863] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:50 2022] DELETE /identity/v3/users/e3440f8c4bc44b8f89070b876dec874d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:50.934248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.936883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.937265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.937375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:50.937771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:50.938194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:50.961534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:50.963055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.034589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] MFA Rules not processed for user `eba035a59725442eb5b39605e06d917d`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:51.036497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.094142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 919/1864] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:51.126329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:51.128398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.150799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6a8e-0cea-406f-a29c-ab0fba1f70ec None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:51.178975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:51.179677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.179959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.180234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] PATH_INFO: `/v3/users/d261890b49e2493884f2f8b6b0f2f574` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.186404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d261890b49e2493884f2f8b6b0f2f574)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:51.188588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:51.194750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 946/1865] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:51.197091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:51.205054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.293287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd261890b49e2493884f2f8b6b0f2f574'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.301873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd261890b49e2493884f2f8b6b0f2f574'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.302585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd261890b49e2493884f2f8b6b0f2f574'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.310062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e005eec-b9a8-4716-a9be-466100acfe5b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:51.319236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae34d3a-8b55-43cf-b7e0-21f44f41d45b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:51.321353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34d3a-8b55-43cf-b7e0-21f44f41d45b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.321470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34d3a-8b55-43cf-b7e0-21f44f41d45b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.322717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae34d3a-8b55-43cf-b7e0-21f44f41d45b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.328252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd261890b49e2493884f2f8b6b0f2f574'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.338503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd261890b49e2493884f2f8b6b0f2f574'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.347574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 920/1866] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:51 2022] DELETE /identity/v3/users/d261890b49e2493884f2f8b6b0f2f574 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:51.348831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:51.351492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.380112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae34d3a-8b55-43cf-b7e0-21f44f41d45b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:51.387347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae34d3a-8b55-43cf-b7e0-21f44f41d45b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:51.425443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 947/1867] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:51.427239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.427536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.427805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.440641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:51.467967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aae60-46ff-46c2-bee4-81de50e9f5a7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:51.487360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:51.488454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:51.488823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.489589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.489920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.490202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.494263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.652706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 948/1868] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:51.668668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:51.671542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8882c9d-9f9d-438e-9ea5-c3bf82eabfca service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:51.686630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 921/1869] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:51.689408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:51.691049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.697176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:51.869146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55013f7e-6e9c-413a-b459-805e8e5eea97 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:51.875511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6110a1-3fc0-4e46-a37e-f38696fa3b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:51.876583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6110a1-3fc0-4e46-a37e-f38696fa3b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.877353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6110a1-3fc0-4e46-a37e-f38696fa3b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.877862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6110a1-3fc0-4e46-a37e-f38696fa3b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.898130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6110a1-3fc0-4e46-a37e-f38696fa3b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:51.903335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6110a1-3fc0-4e46-a37e-f38696fa3b62 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:51.929885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 949/1870] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:52.351588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd24e7-a7bd-46bd-87b4-8f0c27ee0be1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.351888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd24e7-a7bd-46bd-87b4-8f0c27ee0be1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.352211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd24e7-a7bd-46bd-87b4-8f0c27ee0be1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.354257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 922/1871] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:52.424676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.424970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.425241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.483296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:52.483296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:52.606537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 950/1872] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:52.636318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd24e7-a7bd-46bd-87b4-8f0c27ee0be1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:52.640364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7cbd24e7-a7bd-46bd-87b4-8f0c27ee0be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:52.665905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd24e7-a7bd-46bd-87b4-8f0c27ee0be1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:52.673792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709bf2b-2436-4384-84dd-3a91f6fb16d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:52.675579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709bf2b-2436-4384-84dd-3a91f6fb16d5 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.676102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709bf2b-2436-4384-84dd-3a91f6fb16d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.676723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709bf2b-2436-4384-84dd-3a91f6fb16d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.704487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709bf2b-2436-4384-84dd-3a91f6fb16d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:52.706588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709bf2b-2436-4384-84dd-3a91f6fb16d5 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:52.720424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 923/1873] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:52.828056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:52.829656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:52.846514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1046-c370-405a-b80a-eac51f7d3b7e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:52.853077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:52.853833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.854093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.854523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] PATH_INFO: `/v3/projects/cad002877f904e949722bd742b2d7337` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.855761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad002877f904e949722bd742b2d7337)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:52.857966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:52.930214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad002877f904e949722bd742b2d7337', 'request_id': 'req-ae92e651-e664-40ce-a3eb-672add91983d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.944095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad002877f904e949722bd742b2d7337', 'request_id': 'req-ae92e651-e664-40ce-a3eb-672add91983d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.953323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad002877f904e949722bd742b2d7337', 'request_id': 'req-ae92e651-e664-40ce-a3eb-672add91983d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.970577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] The token cache is being invalidate because project cad002877f904e949722bd742b2d7337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:52.970965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.971880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ae92e651-e664-40ce-a3eb-672add91983d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.977974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 951/1874] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:52 2022] DELETE /identity/v3/projects/cad002877f904e949722bd742b2d7337 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:52.979699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eff118-e9c6-4757-b92a-6dd30cf2ccdb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.980171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eff118-e9c6-4757-b92a-6dd30cf2ccdb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.980280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eff118-e9c6-4757-b92a-6dd30cf2ccdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.014416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-22eff118-e9c6-4757-b92a-6dd30cf2ccdb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.016142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-22eff118-e9c6-4757-b92a-6dd30cf2ccdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.094507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 924/1875] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:53.097324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6421fe-3f78-4f27-82dd-5c1e3e790ef3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.097616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6421fe-3f78-4f27-82dd-5c1e3e790ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.097912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6421fe-3f78-4f27-82dd-5c1e3e790ef3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.099427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 952/1876] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:53.102911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.103505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.104043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.108632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.108923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.109191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.135821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.139152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.148311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.149564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.160038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 925/1877] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:53.177621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.182651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.211932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68095e3a-ae8a-4490-a888-adf00f46707b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.222827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.224455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.225200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.225200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.225550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:53.225862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.231870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.254923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 926/1878] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:53.257222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.259510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.278356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862ccbe-60a5-4747-a4e5-768a74daad41 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.279588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 953/1879] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:53.280456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.284148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.284751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.285034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.285311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] PATH_INFO: `/v3/projects/2d6b268c039440c99ad2d9768fe13864` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.290698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6b268c039440c99ad2d9768fe13864)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.292088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2ba4d-e820-434a-92fc-bbec726db37b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.293010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.301679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.302507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.303381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.303381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.303805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.305960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.342194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 954/1880] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:53.342934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.344754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.357761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6b268c039440c99ad2d9768fe13864', 'request_id': 'req-1e032889-1a27-48df-b665-79a8a0b81e70'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.369375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6b268c039440c99ad2d9768fe13864', 'request_id': 'req-1e032889-1a27-48df-b665-79a8a0b81e70'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.371548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.399298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6b268c039440c99ad2d9768fe13864', 'request_id': 'req-1e032889-1a27-48df-b665-79a8a0b81e70'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.415766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] The token cache is being invalidate because project 2d6b268c039440c99ad2d9768fe13864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:53.416178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.416965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.419271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 927/1881] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:53 2022] DELETE /identity/v3/projects/2d6b268c039440c99ad2d9768fe13864 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:53.420098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.421575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.452805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7463bb0-0f1e-43fb-ad8a-faa0337383b2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.460516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.460922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.461217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.461500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.463358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.483129 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.484860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.513200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 955/1882] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:53.514557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.515865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e032889-1a27-48df-b665-79a8a0b81e70 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.516049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.591725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.592404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.592687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.592958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.593672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.595602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.608479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cf92-8fa5-4ed3-b05f-f843d2be0a8a service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.617469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.618161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.618467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.618743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] PATH_INFO: `/v3/users/1d376e81d6e84ed4b0c6c681e1ab2e5c` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.620389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d376e81d6e84ed4b0c6c681e1ab2e5c)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.622364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.626143 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 928/1883] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:53.630131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.644634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3bbe0-25fa-49b1-a6ae-9cf945c034ae admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.648062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.652952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.653825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.654836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.655352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.656356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.658935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.674624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 929/1884] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:53.676223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.688436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad5e59-20a8-4aef-b8b8-8de9cf4a0268 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.693358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d376e81d6e84ed4b0c6c681e1ab2e5c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.694357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.695129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.695452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.695723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.697905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.699538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.703414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d376e81d6e84ed4b0c6c681e1ab2e5c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.720554 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d376e81d6e84ed4b0c6c681e1ab2e5c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.721444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d376e81d6e84ed4b0c6c681e1ab2e5c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.722223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8649d746-fe45-429d-8618-45710e8034dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d376e81d6e84ed4b0c6c681e1ab2e5c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.728393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 930/1885] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:53.733386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.735299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.743567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 956/1886] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:53 2022] DELETE /identity/v3/users/1d376e81d6e84ed4b0c6c681e1ab2e5c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:53.748866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41f86b-12df-417c-8a37-1e56466429d8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.828348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.829021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.829304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.829713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] PATH_INFO: `/v3/projects/1440364a621c4923829c2981476c5d92/users/a2715757eba64780a9c7c45ac560555f/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.832512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.847359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1440364a621c4923829c2981476c5d92, user_id=a2715757eba64780a9c7c45ac560555f, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.850705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.864737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 931/1887] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:53 2022] PUT /identity/v3/projects/1440364a621c4923829c2981476c5d92/users/a2715757eba64780a9c7c45ac560555f/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:53.877056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.877920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.878476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.911453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] MFA Rules not processed for user `a2715757eba64780a9c7c45ac560555f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.912960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.995319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 957/1888] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:53.998838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.069333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.070796 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.075794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bdcba-020b-44db-8b2a-b6d77bd7b259 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.084406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.085724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.086535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.086822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.087723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.088205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33affff2-e09a-49df-8263-efbd8a6c5e87 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.090042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.093847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.094488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.094768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.095117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] PATH_INFO: `/v3/projects/c90af326234145f1b71ec49e84946686` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.096227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90af326234145f1b71ec49e84946686)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.098303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.121074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 932/1889] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:54.125016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.161552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90af326234145f1b71ec49e84946686', 'request_id': 'req-c388fe71-1cc7-4914-8132-101b355e6154'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.171661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90af326234145f1b71ec49e84946686', 'request_id': 'req-c388fe71-1cc7-4914-8132-101b355e6154'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.179077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90af326234145f1b71ec49e84946686', 'request_id': 'req-c388fe71-1cc7-4914-8132-101b355e6154'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.193093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] The token cache is being invalidate because project c90af326234145f1b71ec49e84946686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:54.193491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.194199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c388fe71-1cc7-4914-8132-101b355e6154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.195966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 958/1890] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:54 2022] DELETE /identity/v3/projects/c90af326234145f1b71ec49e84946686 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:54.200899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f725b0-7eba-4961-92da-25bda2ff7da2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.201179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f725b0-7eba-4961-92da-25bda2ff7da2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.201496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f725b0-7eba-4961-92da-25bda2ff7da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.211443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997e57e-96aa-435f-84d8-82c16e5c5498 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.219185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:54.226216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e6f725b0-7eba-4961-92da-25bda2ff7da2 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:54.227603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e6f725b0-7eba-4961-92da-25bda2ff7da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.283367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.284502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.284668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.284950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.285722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.287870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.300091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 959/1891] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:54.307676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.307965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.308276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.314972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 933/1892] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:54.318540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.320277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.328973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:54.333426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:54.334668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.335481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6d69b-59cf-42dd-966a-ac8cac77cfb3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.341922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.342532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.342805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.343294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.344054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.344858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 960/1893] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:54.345957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.350102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.358321 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 934/1894] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:54.362480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61068f1-837b-41f2-af85-87e742530dc5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.363239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.368169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.368795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.369134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.369410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.370197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:54.370469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.372172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.376003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d5c45-ed65-4eb0-9061-5a1c8d625b8c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.381093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.381756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.382086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.382298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] PATH_INFO: `/v3/projects/8773c5029d0f4462a8f9935d97cc361b/users/9e1106d9888641afa7ae402789cbc3f2/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.403267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 961/1895] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:54.403267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.403693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8773c5029d0f4462a8f9935d97cc361b, user_id=9e1106d9888641afa7ae402789cbc3f2, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.405121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.415994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09375c-1bdd-4f11-a7e7-1bd0030c2e7c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.420257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 935/1896] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:54 2022] PUT /identity/v3/projects/8773c5029d0f4462a8f9935d97cc361b/users/9e1106d9888641afa7ae402789cbc3f2/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:54.424294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.521304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.521938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.522221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.522516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.523146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.525704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.527349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612faf26-b32c-42d9-93d4-c043a5d10fb4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.533187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.533808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.534090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.534367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.535069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.536742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.551304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 936/1897] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:54.552593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.559733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 962/1898] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:54.563753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65441ca-7bef-4b1c-9839-41e5bc4a1348 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.567284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.621852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0949c97-def0-4d31-9108-995a96ac71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.622451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949c97-def0-4d31-9108-995a96ac71d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.623346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949c97-def0-4d31-9108-995a96ac71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.623620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949c97-def0-4d31-9108-995a96ac71d9 admin admin] PATH_INFO: `/v3/projects/8773c5029d0f4462a8f9935d97cc361b/users/9e1106d9888641afa7ae402789cbc3f2/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.625520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949c97-def0-4d31-9108-995a96ac71d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8773c5029d0f4462a8f9935d97cc361b, user_id=9e1106d9888641afa7ae402789cbc3f2, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.628384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949c97-def0-4d31-9108-995a96ac71d9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.631764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37a9bc-23db-40c8-a906-0227237ef9c4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.637155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.637768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.638592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.638592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.639062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 937/1899] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:54 2022] PUT /identity/v3/projects/8773c5029d0f4462a8f9935d97cc361b/users/9e1106d9888641afa7ae402789cbc3f2/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:54.646529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.648579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.649446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.649812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.650177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.672234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 963/1900] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:54.676907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.682870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] MFA Rules not processed for user `9e1106d9888641afa7ae402789cbc3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:54.683475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:54.684973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.690244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795d32-598a-4a83-9668-17e3e27e689c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.742590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:54.759363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.760521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.761475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.761754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.762404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.764151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.771800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 938/1901] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:54.778857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 964/1902] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:54.780435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.790741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.797061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.800673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40c17-c26f-4a96-991a-11c7bb2771aa None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.805517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.806137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.806425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.806690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.807365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.809064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.823797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 939/1903] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:54.825301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.836462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3d59-31a4-425c-a73d-697cabede9ec admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.843016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fabaf-0559-4915-84f8-98a400921b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.844001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fabaf-0559-4915-84f8-98a400921b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.844559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fabaf-0559-4915-84f8-98a400921b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.844769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fabaf-0559-4915-84f8-98a400921b06 admin admin] PATH_INFO: `/v3/projects/1029216b9424416e808407c8d5a7afee/users/3b81670147f24e5aa2d2200440d516ef/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.863614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fabaf-0559-4915-84f8-98a400921b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1029216b9424416e808407c8d5a7afee, user_id=3b81670147f24e5aa2d2200440d516ef, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.878026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fabaf-0559-4915-84f8-98a400921b06 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.879330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769790b-9b6f-47d5-a3f2-9817a206b3b7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.889085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 940/1904] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:54 2022] PUT /identity/v3/projects/1029216b9424416e808407c8d5a7afee/users/3b81670147f24e5aa2d2200440d516ef/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:54.907546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.908088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.908542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.946450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] MFA Rules not processed for user `3b81670147f24e5aa2d2200440d516ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:54.948115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.963489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.964366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.964994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.965271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.018791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 941/1905] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:55.023325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.031068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.032742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.042528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 965/1906] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:55.093706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74284aec-8f7c-4489-a240-4a7a10a2dcb0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.099149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.099869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.100362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.100678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.101517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.104065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.136044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.141028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 942/1907] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:55.141956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.145091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.145952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ee8ed-56c7-4041-8d4f-07a4812a37b2 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.180460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.219522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.220402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.220837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.221260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.223425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.238984 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cb561-4706-46ac-a48e-8156af55660e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.244145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.244786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.245071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.245346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.246003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.248234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.276774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 943/1908] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:55 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:55.281370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.294840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435eb3e-a159-4dbe-8a97-2ef65ad19825 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.299550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.300242 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.300597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.300953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.301672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.303470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.306763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.308253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.313925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 944/1909] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:55.317786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 966/1910] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:55.318398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.329106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0fd1-fb8d-416d-8047-e1de20c6b25b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.334225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.334820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.335104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.335422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] PATH_INFO: `/v3/projects/ee83996eeb604d1e9591a06560d3e2ef/users/dbd6ccefa1be4290a30825daf0c91cab/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.351965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee83996eeb604d1e9591a06560d3e2ef, user_id=dbd6ccefa1be4290a30825daf0c91cab, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.354789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.371164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 945/1911] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:55 2022] PUT /identity/v3/projects/ee83996eeb604d1e9591a06560d3e2ef/users/dbd6ccefa1be4290a30825daf0c91cab/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:55.376318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.389159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a705fd-a2b4-49b3-905c-8e3cfed19dc9 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.465566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab91ea-5587-4695-af99-b8a4226ff35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.466465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab91ea-5587-4695-af99-b8a4226ff35e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.467405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab91ea-5587-4695-af99-b8a4226ff35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.467847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab91ea-5587-4695-af99-b8a4226ff35e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.468593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab91ea-5587-4695-af99-b8a4226ff35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.470308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab91ea-5587-4695-af99-b8a4226ff35e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.484016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 967/1912] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:55.490331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.507924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40b92-5a34-45b5-815c-3c18a7d1b083 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.524359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.524359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.524359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.524359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] PATH_INFO: `/v3/projects/ee83996eeb604d1e9591a06560d3e2ef/users/dbd6ccefa1be4290a30825daf0c91cab/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.524704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee83996eeb604d1e9591a06560d3e2ef, user_id=dbd6ccefa1be4290a30825daf0c91cab, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.524904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.542325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 946/1913] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:55 2022] PUT /identity/v3/projects/ee83996eeb604d1e9591a06560d3e2ef/users/dbd6ccefa1be4290a30825daf0c91cab/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:55.552422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.552717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.553002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.587916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] MFA Rules not processed for user `dbd6ccefa1be4290a30825daf0c91cab`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:55.589420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.650395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.675257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 968/1914] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:55.686655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.740301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aef8-0c10-42df-8752-515746f8b225 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.743779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.744375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.744652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.744918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.754293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.755745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.766229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 947/1915] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:55.870201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.880475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe498c8-0a22-4586-b43d-70cf47f7f23e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.884194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.884807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.885086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.885617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.948137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.949593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.959588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 969/1916] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:58.952651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.954098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.965099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a5949-3dd0-421c-b2a3-40372e6341ea service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.969510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.970106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.970388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.970657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.972546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:59.038950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.040471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.050001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 948/1917] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:59.898539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.985081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.994155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b665c1e-bb13-48f4-a615-e8101304f401 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.000258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.001109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.001444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.005158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.005158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.007172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe08dc-30c3-4607-a503-5086fef8fcc5 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.010882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:00.013153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.013780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.014068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.014384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] PATH_INFO: `/v3/users/3b81670147f24e5aa2d2200440d516ef` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.015574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b81670147f24e5aa2d2200440d516ef)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.017555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.086638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b81670147f24e5aa2d2200440d516ef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.096818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b81670147f24e5aa2d2200440d516ef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.102643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.104369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c584a-f251-433f-bd32-e100f36a1af1 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.114010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b81670147f24e5aa2d2200440d516ef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.114869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b81670147f24e5aa2d2200440d516ef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.115606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b81670147f24e5aa2d2200440d516ef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.116614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 949/1918] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:00.126722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:00.132522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 970/1919] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:59 2022] DELETE /identity/v3/users/3b81670147f24e5aa2d2200440d516ef => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:00.138233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.138585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.139735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.163550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:00.164752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.264327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 950/1920] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:00.265435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:00.266816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.325740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639c12-f220-46d8-8c5e-ae0540b0f329 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.329360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.329968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.330320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.330551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.340088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.342035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.352283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 971/1921] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:00.637285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:00.639307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.651691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f54de-35d8-4b7d-bec5-80a92dd687ee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.663741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.664534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.664827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.665103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] PATH_INFO: `/v3/projects/1029216b9424416e808407c8d5a7afee` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.666301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1029216b9424416e808407c8d5a7afee)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.669032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.736921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1029216b9424416e808407c8d5a7afee', 'request_id': 'req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.748448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1029216b9424416e808407c8d5a7afee', 'request_id': 'req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.757945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1029216b9424416e808407c8d5a7afee', 'request_id': 'req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.770528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] The token cache is being invalidate because project 1029216b9424416e808407c8d5a7afee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:00.770824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.771520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.773294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 951/1922] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:00 2022] DELETE /identity/v3/projects/1029216b9424416e808407c8d5a7afee => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:00.777074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:00.794379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef3778-24a4-49e0-b410-ae588e1a765a service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.858604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.859344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.859715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.859983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] PATH_INFO: `/v3/users/dbd6ccefa1be4290a30825daf0c91cab` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.861196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd6ccefa1be4290a30825daf0c91cab)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.863191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.921646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd6ccefa1be4290a30825daf0c91cab'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.929976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd6ccefa1be4290a30825daf0c91cab'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.945137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd6ccefa1be4290a30825daf0c91cab'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.946005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd6ccefa1be4290a30825daf0c91cab'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.946918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-533b2836-acde-483b-8775-5bff4ae07e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd6ccefa1be4290a30825daf0c91cab'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.965125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 972/1923] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:00 2022] DELETE /identity/v3/users/dbd6ccefa1be4290a30825daf0c91cab => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:01.292711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.295695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:01.327104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a535eea-47e4-4ff0-ad46-6c59eeb75fcb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.360325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:01.396023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.396660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.396934 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.397203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] PATH_INFO: `/v3/projects/ee83996eeb604d1e9591a06560d3e2ef` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.398408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee83996eeb604d1e9591a06560d3e2ef)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.400253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.446446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee83996eeb604d1e9591a06560d3e2ef', 'request_id': 'req-05c7b748-bbf3-4857-85cd-5e001157468d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:01.454875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee83996eeb604d1e9591a06560d3e2ef', 'request_id': 'req-05c7b748-bbf3-4857-85cd-5e001157468d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:01.460420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee83996eeb604d1e9591a06560d3e2ef', 'request_id': 'req-05c7b748-bbf3-4857-85cd-5e001157468d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:01.476049 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] The token cache is being invalidate because project ee83996eeb604d1e9591a06560d3e2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:01.476341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:01.477508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-05c7b748-bbf3-4857-85cd-5e001157468d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:01.479360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 952/1924] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:01 2022] DELETE /identity/v3/projects/ee83996eeb604d1e9591a06560d3e2ef => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:01.485321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.485610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.485878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.507757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:01.509068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:01.525071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:01.593697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 973/1925] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:01.595484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.595822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.596046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.617077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:01.618327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:01.626714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 953/1926] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:01.631341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.641679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8660-4e35-4a8f-9110-d8f705387bc1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.646286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.646967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.647160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.647465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.648106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:01.648407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.650041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.662059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 974/1927] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:01.666494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.676612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf2a3f3-5bad-4b09-b5e4-d5648807347d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.681279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.681896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.682176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.682444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.683079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.685056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.712889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 954/1928] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:01.717942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.728259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eacac9-e1ee-4da5-ba9c-519b2e120f1d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.789571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.790218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.790499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.790767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.791453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.793505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.816805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 975/1929] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:01.821210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.832382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a6127-1e82-41a4-931f-ffaad051c08b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.837280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.837902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.838182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.838455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.839099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.840797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.851292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 955/1930] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:01.856453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.873765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f2e95-61a8-4b07-b469-5db34249b3a7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.878437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd542ab8-33db-4e37-a876-0d067a8d8164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.879054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542ab8-33db-4e37-a876-0d067a8d8164 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.879361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542ab8-33db-4e37-a876-0d067a8d8164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.879710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542ab8-33db-4e37-a876-0d067a8d8164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.880355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd542ab8-33db-4e37-a876-0d067a8d8164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.881950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd542ab8-33db-4e37-a876-0d067a8d8164 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.894374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 976/1931] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:01.898638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.909134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8486c-ced0-455d-9a29-c822a6361342 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.913837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.914525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.914803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.915077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] PATH_INFO: `/v3/projects/198470c2029a4ef6bea8dbbbb87201f7/users/911c3d3126ea4256b8e09c6147e406ed/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.931534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] RBAC: Authorizing `identity:create_grant(project_id=198470c2029a4ef6bea8dbbbb87201f7, user_id=911c3d3126ea4256b8e09c6147e406ed, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.934192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.948728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 956/1932] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:01 2022] PUT /identity/v3/projects/198470c2029a4ef6bea8dbbbb87201f7/users/911c3d3126ea4256b8e09c6147e406ed/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:01.960723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.961248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.961394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.991742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] MFA Rules not processed for user `911c3d3126ea4256b8e09c6147e406ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:01.993113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:02.056874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 977/1933] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:02.062044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.121015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:02.139522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91431c9a-c0bd-4e94-a7cd-9f06bc5c858d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.145166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.145816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.146097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.146537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.147182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.149124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.178742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 957/1934] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:02.182801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.195361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9828a1e-50b8-4cd8-88ee-a6706dd38336 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.208343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:02.256445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.257100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.257809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.257809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.258257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.260494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.283357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 978/1935] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:02.286713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.297609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3ec11-075e-4a90-9729-8d2ab9b0a698 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.302557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.303160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.303474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.303752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.304401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.305999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.320637 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 958/1936] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:02.326072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.336667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deecf80-caff-4491-963e-0981a7484082 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.342590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.343295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.343690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.344036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] PATH_INFO: `/v3/projects/e49e5a9b334a47f7b6a5aec6d50924b8/users/3adda353ef024f5a911ac0d955abd7d8/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.361649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49e5a9b334a47f7b6a5aec6d50924b8, user_id=3adda353ef024f5a911ac0d955abd7d8, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.364389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.380385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 979/1937] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:02 2022] PUT /identity/v3/projects/e49e5a9b334a47f7b6a5aec6d50924b8/users/3adda353ef024f5a911ac0d955abd7d8/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:02.384413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.394933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cce02e-e0bc-46d0-9af8-24b28aac604d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.470997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36299-c485-4eb3-9766-3f266fff05b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.471739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36299-c485-4eb3-9766-3f266fff05b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.472128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36299-c485-4eb3-9766-3f266fff05b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.472470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36299-c485-4eb3-9766-3f266fff05b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.473266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36299-c485-4eb3-9766-3f266fff05b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.474957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36299-c485-4eb3-9766-3f266fff05b1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.485620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 959/1938] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:02.489948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.500285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57291d-12c0-475c-a2a4-63ab76c12239 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.505056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.505648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.505930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.506206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] PATH_INFO: `/v3/projects/e49e5a9b334a47f7b6a5aec6d50924b8/users/3adda353ef024f5a911ac0d955abd7d8/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.508544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49e5a9b334a47f7b6a5aec6d50924b8, user_id=3adda353ef024f5a911ac0d955abd7d8, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.511229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.522452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 980/1939] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:02 2022] PUT /identity/v3/projects/e49e5a9b334a47f7b6a5aec6d50924b8/users/3adda353ef024f5a911ac0d955abd7d8/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:02.533285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.533574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.533852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.561645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] MFA Rules not processed for user `3adda353ef024f5a911ac0d955abd7d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:02.562860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:02.629000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 960/1940] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:02.639955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.641680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:02.702131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f2099-adab-48c9-a340-43db1bd9ac64 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.705867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6dd62-2097-4bbc-8e54-33d1ed4570b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.706479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6dd62-2097-4bbc-8e54-33d1ed4570b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.706754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6dd62-2097-4bbc-8e54-33d1ed4570b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.707023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6dd62-2097-4bbc-8e54-33d1ed4570b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.717127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6dd62-2097-4bbc-8e54-33d1ed4570b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.718563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6dd62-2097-4bbc-8e54-33d1ed4570b2 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.732647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 981/1941] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:03.156334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:03.205141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:03.222807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e63c2-8ea0-4fb8-b439-d9fb14125428 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:03.227558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:03.228201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:03.228583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:03.228864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] PATH_INFO: `/v3/users/911c3d3126ea4256b8e09c6147e406ed` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:03.230025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=911c3d3126ea4256b8e09c6147e406ed)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:03.232063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:03.279848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911c3d3126ea4256b8e09c6147e406ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.287706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911c3d3126ea4256b8e09c6147e406ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.288325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911c3d3126ea4256b8e09c6147e406ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.301427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911c3d3126ea4256b8e09c6147e406ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.310788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911c3d3126ea4256b8e09c6147e406ed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.316773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 961/1942] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:03 2022] DELETE /identity/v3/users/911c3d3126ea4256b8e09c6147e406ed => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:03.321491 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:03.321946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:03.322087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:03.342350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:03.343756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:03.420007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 982/1943] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:03.430986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:03.432622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:03.494373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596d5c0-8f23-4b95-bd15-2d8eb23b5ad7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:03.498978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:03.499412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:03.499734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:03.500068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:03.514641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:03.518054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:03.530594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 962/1944] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:03.805597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:03.807372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:03.823798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd4bcf-d148-421e-8fbc-2780af3766d7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:03.829159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:03.829885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:03.830198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:03.830478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] PATH_INFO: `/v3/projects/198470c2029a4ef6bea8dbbbb87201f7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:03.831660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=198470c2029a4ef6bea8dbbbb87201f7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:03.833591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:03.840511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:03.890027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198470c2029a4ef6bea8dbbbb87201f7', 'request_id': 'req-3309a06c-100c-4a96-8712-a6d52195c4f6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.898951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198470c2029a4ef6bea8dbbbb87201f7', 'request_id': 'req-3309a06c-100c-4a96-8712-a6d52195c4f6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.908314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198470c2029a4ef6bea8dbbbb87201f7', 'request_id': 'req-3309a06c-100c-4a96-8712-a6d52195c4f6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.923566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] The token cache is being invalidate because project 198470c2029a4ef6bea8dbbbb87201f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:03.924308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.924540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:03.926857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 983/1945] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:03 2022] DELETE /identity/v3/projects/198470c2029a4ef6bea8dbbbb87201f7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:03.930514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:03.941477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a0bb1-2f61-486e-8c60-cd0f083ccd1a service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:04.020977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:04.021812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.022052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.022427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] PATH_INFO: `/v3/users/3adda353ef024f5a911ac0d955abd7d8` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.023714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3adda353ef024f5a911ac0d955abd7d8)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:04.025826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:04.038640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:04.085958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3adda353ef024f5a911ac0d955abd7d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.101056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3adda353ef024f5a911ac0d955abd7d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.101916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3adda353ef024f5a911ac0d955abd7d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.116653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3adda353ef024f5a911ac0d955abd7d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.125795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6ef1c42c-7837-4546-8a78-1caed4a8d849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3adda353ef024f5a911ac0d955abd7d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.132376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 963/1946] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:03 2022] DELETE /identity/v3/users/3adda353ef024f5a911ac0d955abd7d8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:04.440432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:04.444162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:04.459318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309a06c-100c-4a96-8712-a6d52195c4f6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:04.558819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:04.559516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.560476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.560638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] PATH_INFO: `/v3/projects/e49e5a9b334a47f7b6a5aec6d50924b8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.561897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e49e5a9b334a47f7b6a5aec6d50924b8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:04.564044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:04.626972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e49e5a9b334a47f7b6a5aec6d50924b8', 'request_id': 'req-3481f438-9955-4b7a-82cf-5746dcdcfe4d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.635873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e49e5a9b334a47f7b6a5aec6d50924b8', 'request_id': 'req-3481f438-9955-4b7a-82cf-5746dcdcfe4d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.644620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e49e5a9b334a47f7b6a5aec6d50924b8', 'request_id': 'req-3481f438-9955-4b7a-82cf-5746dcdcfe4d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.656560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] The token cache is being invalidate because project e49e5a9b334a47f7b6a5aec6d50924b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:04.657002 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.657668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3481f438-9955-4b7a-82cf-5746dcdcfe4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:04.659183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 984/1947] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:04 2022] DELETE /identity/v3/projects/e49e5a9b334a47f7b6a5aec6d50924b8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:04.665146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.665645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.665731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.688009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:04.689306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:04.755048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 964/1948] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:04.762943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.763251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.763531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.783575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:04.784784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:04.792598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 985/1949] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:04.797623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:04.807408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fc012-ae2a-492e-9c9f-71121bfc84ee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:04.812653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:04.813263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.813545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.813815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.814463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:04.814766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:04.816438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:04.828325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 965/1950] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:04.832706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:04.842876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be8f26-2877-4384-8828-e28a47b3859d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:04.847938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:04.848544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.848822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.849104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.849762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:04.851722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:04.868085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:04.881694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 986/1951] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:04 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:04.886340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:04.896584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9a600-ad2a-428e-989a-f0ca70272092 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:04.954836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:04.955488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:04.955769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:04.956043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:04.956685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:04.958629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:04.981365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 966/1952] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:04 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:04.986239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:04.997205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5c94-0303-4836-9797-d5cc34e40cab admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.002227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.002854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.003134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.003512 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.004222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.005900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.017518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 987/1953] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:05.022350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.028212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:05.033131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd7f0-b226-4b6c-ad8b-514d6d0de5ea admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.040172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aea1e-5aa1-43e6-b248-1dedaea123da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.041196 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aea1e-5aa1-43e6-b248-1dedaea123da admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.041789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aea1e-5aa1-43e6-b248-1dedaea123da admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.041952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aea1e-5aa1-43e6-b248-1dedaea123da admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.042626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aea1e-5aa1-43e6-b248-1dedaea123da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.045129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aea1e-5aa1-43e6-b248-1dedaea123da admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.056742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 967/1954] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:05.061188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.071556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bba28-ccc0-4a29-9cc1-f366c8fa735a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.076869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.077544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.077823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.078102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] PATH_INFO: `/v3/projects/9220968e3b18463cbddf0123a20427f0/users/6734b944f3c5434985b879e5bae92fe3/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.094600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9220968e3b18463cbddf0123a20427f0, user_id=6734b944f3c5434985b879e5bae92fe3, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.097338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.112449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 988/1955] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:05 2022] PUT /identity/v3/projects/9220968e3b18463cbddf0123a20427f0/users/6734b944f3c5434985b879e5bae92fe3/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:05.123808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05290e48-6355-4434-85ef-61cf383b391d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.124097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05290e48-6355-4434-85ef-61cf383b391d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.124376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05290e48-6355-4434-85ef-61cf383b391d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.154873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-05290e48-6355-4434-85ef-61cf383b391d None None] MFA Rules not processed for user `6734b944f3c5434985b879e5bae92fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:05.156148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-05290e48-6355-4434-85ef-61cf383b391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:05.224950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 968/1956] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:05.229619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.292146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eda3a7-6046-4a1c-92b9-dbdb4eb8b9a9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.296977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.297589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.297869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.298145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.298794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.300758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.328151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 989/1957] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:05 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:05.332147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05290e48-6355-4434-85ef-61cf383b391d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.343517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05290e48-6355-4434-85ef-61cf383b391d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.429296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.430653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.431460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.432441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.434275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.438504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.484074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 969/1958] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:05 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:05.487876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.499806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360ceae-9b1d-4735-adce-b9e352d7875f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.505093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.506338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.506718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.507087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.507881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.509981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.528294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 990/1959] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:05.529387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.542979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd748b10-0952-4a21-8041-c53ab7353a27 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.551986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760c78c-cfb6-44cc-9d95-cd9bef22b9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.552748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760c78c-cfb6-44cc-9d95-cd9bef22b9fc admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.553154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760c78c-cfb6-44cc-9d95-cd9bef22b9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.553528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760c78c-cfb6-44cc-9d95-cd9bef22b9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.554298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760c78c-cfb6-44cc-9d95-cd9bef22b9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.556365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760c78c-cfb6-44cc-9d95-cd9bef22b9fc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.568922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 970/1960] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:05.573199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.588901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae48f4-e73c-4e50-b990-d7461d89a16a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.596134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.597076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.597501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.597913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] PATH_INFO: `/v3/projects/805be4f47d064d81a769c8ee3a61c4fd/users/751ecd963f7c482799233e379d7312a1/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.619058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805be4f47d064d81a769c8ee3a61c4fd, user_id=751ecd963f7c482799233e379d7312a1, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.622433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.652307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 991/1961] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:05 2022] PUT /identity/v3/projects/805be4f47d064d81a769c8ee3a61c4fd/users/751ecd963f7c482799233e379d7312a1/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:05.655759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.656178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.656831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.690832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] MFA Rules not processed for user `751ecd963f7c482799233e379d7312a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:05.692288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:05.753125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:05.779962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 971/1962] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:05.784231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.878999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba960c6a-47b4-42ae-a06c-34cbcfe34eb0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.883961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.884586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.884863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.885387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.886049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.888529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:05.905291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:05.921855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 992/1963] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:05 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:05.923000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:05.934305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac34255-98bd-4ff9-ae27-7e5fe9347cee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:05.993547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:05.994165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:05.994449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:05.994723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:05.995399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:05.997324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.020918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 972/1964] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:05 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:06.025182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.036752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fe610-6fac-4078-8e8b-a192a82919fa admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.042181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.042833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.043094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.043441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.044177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.045870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.057203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 993/1965] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:06.061593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.071789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d82cb-8a7d-45e3-9aa5-e36718f10f01 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.077530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.078192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.078472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.078749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] PATH_INFO: `/v3/projects/2ad670ff862940439fe89608996e6660/users/ec5db64a317842d993ff99ebbd711969/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.096190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad670ff862940439fe89608996e6660, user_id=ec5db64a317842d993ff99ebbd711969, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.103216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.128080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 973/1966] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:06 2022] PUT /identity/v3/projects/2ad670ff862940439fe89608996e6660/users/ec5db64a317842d993ff99ebbd711969/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:06.132265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.146081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104141d3-097f-41db-b6e0-3cf7c32b2a3e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.204521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547e24-b983-4f17-abea-7dd04e6df55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.205136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547e24-b983-4f17-abea-7dd04e6df55c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.205435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547e24-b983-4f17-abea-7dd04e6df55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.205714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547e24-b983-4f17-abea-7dd04e6df55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.206358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87547e24-b983-4f17-abea-7dd04e6df55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.208013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87547e24-b983-4f17-abea-7dd04e6df55c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.218323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 994/1967] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:06.222763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.233573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb512a-1049-44f9-a247-6b45a5c440f6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.238382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.239047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.239361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.239644 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] PATH_INFO: `/v3/projects/2ad670ff862940439fe89608996e6660/users/ec5db64a317842d993ff99ebbd711969/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.241593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad670ff862940439fe89608996e6660, user_id=ec5db64a317842d993ff99ebbd711969, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.244311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.255656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 974/1968] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:06 2022] PUT /identity/v3/projects/2ad670ff862940439fe89608996e6660/users/ec5db64a317842d993ff99ebbd711969/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:06.268233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b077b6f-e2b1-4d98-a613-805c1c1a878e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.268519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b077b6f-e2b1-4d98-a613-805c1c1a878e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.268795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b077b6f-e2b1-4d98-a613-805c1c1a878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.298758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0b077b6f-e2b1-4d98-a613-805c1c1a878e None None] MFA Rules not processed for user `ec5db64a317842d993ff99ebbd711969`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:06.300052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0b077b6f-e2b1-4d98-a613-805c1c1a878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.385843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 995/1969] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:06.397832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.399605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.465267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276793-598c-409d-bcf8-18fbb6a3fc37 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.468993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61175d9-7268-40d8-b7d3-147e2cfe39e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.469602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61175d9-7268-40d8-b7d3-147e2cfe39e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.469879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61175d9-7268-40d8-b7d3-147e2cfe39e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.470149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61175d9-7268-40d8-b7d3-147e2cfe39e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.535445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61175d9-7268-40d8-b7d3-147e2cfe39e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.536869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61175d9-7268-40d8-b7d3-147e2cfe39e7 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.542089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:06.547101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 975/1970] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:10.631934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001cfd4-28b6-4b9a-8182-de16de1063d6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.632788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001cfd4-28b6-4b9a-8182-de16de1063d6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.633209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001cfd4-28b6-4b9a-8182-de16de1063d6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.634737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 996/1971] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 14:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:59:10.643761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.644317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.644618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.666479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:10.667529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.757189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 976/1972] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:10.770327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001cfd4-28b6-4b9a-8182-de16de1063d6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.772088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0001cfd4-28b6-4b9a-8182-de16de1063d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.791978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001cfd4-28b6-4b9a-8182-de16de1063d6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.797425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.798031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.798434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.798713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.808260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.809869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.821287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 997/1973] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:13.463495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.465986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.478378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9245a5f-eea3-43bc-814e-3bb179ce1d72 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.490174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.490722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.491000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.491312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.564363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.565988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.575703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 977/1974] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:17.420233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:17.431016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a766c-88d5-4dbc-a704-3f232828cff4 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:17.435073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bec45-34c4-466e-9269-6b1474d3c120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:17.435681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bec45-34c4-466e-9269-6b1474d3c120 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:17.435983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bec45-34c4-466e-9269-6b1474d3c120 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:17.436320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bec45-34c4-466e-9269-6b1474d3c120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:17.445747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bec45-34c4-466e-9269-6b1474d3c120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:17.447237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bec45-34c4-466e-9269-6b1474d3c120 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:17.457333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 998/1975] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:20.429351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:20.430988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:20.508938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e946-480b-4b4a-8d0f-854885a58213 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:20.513931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:20.514585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:20.514862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:20.515131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] PATH_INFO: `/v3/users/b5916793c840454a9aab5bfdd46fea9d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:20.516327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5916793c840454a9aab5bfdd46fea9d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:20.518273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:20.556064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:20.571081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5916793c840454a9aab5bfdd46fea9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.579594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5916793c840454a9aab5bfdd46fea9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.580288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5916793c840454a9aab5bfdd46fea9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.594352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5916793c840454a9aab5bfdd46fea9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.601529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5916793c840454a9aab5bfdd46fea9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.610545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 978/1976] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:20 2022] DELETE /identity/v3/users/b5916793c840454a9aab5bfdd46fea9d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:20.615385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:20.616156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:20.616453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:20.636120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:20.637433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:20.689095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:20.718758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 999/1977] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:20.728927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:20.730368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:20.789518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bf364-eff3-4cf7-b3e4-3c9d76a8fb80 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:20.793189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:20.793725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:20.793974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:20.794241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:20.803929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:20.805372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:20.814802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 979/1978] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:21.180737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.182382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:21.216194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b9ee1-469d-496c-8fd7-3e4d595dc8cd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.223822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.224602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.225094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.225436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] PATH_INFO: `/v3/projects/e343f78042c0450794e592f1ea1abc4a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.226595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e343f78042c0450794e592f1ea1abc4a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.228629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.288567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e343f78042c0450794e592f1ea1abc4a', 'request_id': 'req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.304076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e343f78042c0450794e592f1ea1abc4a', 'request_id': 'req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.311812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e343f78042c0450794e592f1ea1abc4a', 'request_id': 'req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.328875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] The token cache is being invalidate because project e343f78042c0450794e592f1ea1abc4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:21.329184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.329931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.332256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1000/1979] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:21 2022] DELETE /identity/v3/projects/e343f78042c0450794e592f1ea1abc4a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:21.336510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.347302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654032be-fca4-4a30-a788-9e6abb9b53f2 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.409518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.410159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.410444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.410715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] PATH_INFO: `/v3/users/036542e3801c4f148ab45391ba57db2d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.411881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=036542e3801c4f148ab45391ba57db2d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.413833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.460968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036542e3801c4f148ab45391ba57db2d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.464565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:21.469175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036542e3801c4f148ab45391ba57db2d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.470305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036542e3801c4f148ab45391ba57db2d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.485448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036542e3801c4f148ab45391ba57db2d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.492530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036542e3801c4f148ab45391ba57db2d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.499263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 980/1980] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:21 2022] DELETE /identity/v3/users/036542e3801c4f148ab45391ba57db2d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:21.839963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.841722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:21.857382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0723-c70b-487c-9e4f-b2b0f97f7cd5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.920058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.921265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.921543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.921812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] PATH_INFO: `/v3/projects/16b90495aaf34a308d48291e664baa12` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.922942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b90495aaf34a308d48291e664baa12)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.924895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.972028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b90495aaf34a308d48291e664baa12', 'request_id': 'req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.980737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b90495aaf34a308d48291e664baa12', 'request_id': 'req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.988485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b90495aaf34a308d48291e664baa12', 'request_id': 'req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.989583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:22.004991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] The token cache is being invalidate because project 16b90495aaf34a308d48291e664baa12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:22.005431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.006328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.007925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1001/1981] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:21 2022] DELETE /identity/v3/projects/16b90495aaf34a308d48291e664baa12 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:22.012266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.013796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.026587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15a8c2-11a0-4670-8ce9-c7db12f560ca admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.086538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.087175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.087491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.087763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] PATH_INFO: `/v3/users/eba035a59725442eb5b39605e06d917d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.088898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba035a59725442eb5b39605e06d917d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.090826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.141318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba035a59725442eb5b39605e06d917d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.149882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba035a59725442eb5b39605e06d917d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.150588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba035a59725442eb5b39605e06d917d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.166952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba035a59725442eb5b39605e06d917d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.180024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-491613d8-05ba-4888-a5e9-85bdff2c781d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba035a59725442eb5b39605e06d917d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.186350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 981/1982] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:22 2022] DELETE /identity/v3/users/eba035a59725442eb5b39605e06d917d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:22.630622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.632587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.647105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fda4b-8b5e-4b0c-a1d2-21e4e0976f5e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.715655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.716303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.716595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.716865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] PATH_INFO: `/v3/projects/f99fffebf6874d8baefd12f3571c33a0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.718105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99fffebf6874d8baefd12f3571c33a0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.720184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.773197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99fffebf6874d8baefd12f3571c33a0', 'request_id': 'req-699504ab-9ee9-47c9-a55d-1424231de884'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.782326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99fffebf6874d8baefd12f3571c33a0', 'request_id': 'req-699504ab-9ee9-47c9-a55d-1424231de884'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.790386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99fffebf6874d8baefd12f3571c33a0', 'request_id': 'req-699504ab-9ee9-47c9-a55d-1424231de884'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.802078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] The token cache is being invalidate because project f99fffebf6874d8baefd12f3571c33a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:22.802360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.803030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-699504ab-9ee9-47c9-a55d-1424231de884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.804594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1002/1983] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:22 2022] DELETE /identity/v3/projects/f99fffebf6874d8baefd12f3571c33a0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:22.810087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.810542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.810696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.831621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:22.832833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.882306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:22.902048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 982/1984] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:22.910193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.910676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.910786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.931968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:22.933207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.941710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1003/1985] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:22.946057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.955836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ec519-a864-4ae7-b6c7-9c865dbb6548 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.960418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.961015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.961299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.961574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.962215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:22.962515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.964190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.975795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 983/1986] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:22.980242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.989886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e6f48-0bae-4840-9559-b4b523571ade None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.994457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.995051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.995374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.995648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.996273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.998178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.026207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1004/1987] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:22 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:23.030335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.040194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e71d2-1ada-4ae1-ac96-6dbff39a1ace admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.095491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.096129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.096413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.096686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.097384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.099341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.122872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 984/1988] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:23 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:23.127733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.139710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac3f73-6c4a-4348-bae5-87b2b9d5d298 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.144435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.145057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.145333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.145597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.146242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.147899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.158805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1005/1989] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:23.164063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.175407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719367-8644-475c-8e2e-eaddbe3a889f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.180463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa1eaa-07c8-43ca-9438-017d2cb834e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.181190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1eaa-07c8-43ca-9438-017d2cb834e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.181531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1eaa-07c8-43ca-9438-017d2cb834e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.181817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1eaa-07c8-43ca-9438-017d2cb834e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.182476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa1eaa-07c8-43ca-9438-017d2cb834e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.184164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa1eaa-07c8-43ca-9438-017d2cb834e0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.196305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 985/1990] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:23.200992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.212936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ddc64-be05-438c-a5b8-706d19d0b9cf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.217413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.218086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.218364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.218642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] PATH_INFO: `/v3/projects/b6e34cb528274bc88d6d1a99d8e0ca11/users/f0e01022f97340a8b8ee0a17bb451df7/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.235259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e34cb528274bc88d6d1a99d8e0ca11, user_id=f0e01022f97340a8b8ee0a17bb451df7, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.238242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.255243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1006/1991] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:23 2022] PUT /identity/v3/projects/b6e34cb528274bc88d6d1a99d8e0ca11/users/f0e01022f97340a8b8ee0a17bb451df7/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:23.265648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.265935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.266307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.294752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] MFA Rules not processed for user `f0e01022f97340a8b8ee0a17bb451df7`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:23.296371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:23.366422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 986/1992] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:23.378370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.380281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:23.402065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:23.447921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009cfb8-c0e9-4c63-9f34-2a51c007e1db admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.452995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.453667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.453953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.454233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.463182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.464658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.487651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1007/1993] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:33.363515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:33.475922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=17 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:33.895950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:33.897793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:33.976575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca23f09-0ea4-43cd-93c1-b921f5ec1494 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:33.985572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:33.986672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:33.987059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:33.987533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] PATH_INFO: `/v3/users/a2715757eba64780a9c7c45ac560555f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:33.990092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2715757eba64780a9c7c45ac560555f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:33.993239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.006472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.008285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.033915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9b616-40bc-47be-a4f0-d2a754e88d8d service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:34.040797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:34.041821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.042655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.043351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] PATH_INFO: `/v3/users/bece5c63200640438821a7ef86709ed5` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.045022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bece5c63200640438821a7ef86709ed5)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:34.048770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.072440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2715757eba64780a9c7c45ac560555f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.084425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2715757eba64780a9c7c45ac560555f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.085168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2715757eba64780a9c7c45ac560555f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.103898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2715757eba64780a9c7c45ac560555f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.111394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11151156-33af-4513-9385-8d76f9e374ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2715757eba64780a9c7c45ac560555f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.115749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bece5c63200640438821a7ef86709ed5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.118243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 987/1994] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:33 2022] DELETE /identity/v3/users/a2715757eba64780a9c7c45ac560555f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:34.123061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.123817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.124682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.131171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bece5c63200640438821a7ef86709ed5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.160043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:34.161694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.164021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bece5c63200640438821a7ef86709ed5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.164879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bece5c63200640438821a7ef86709ed5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.165859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f169718-c6f7-4bcf-90d0-4b03c3270b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bece5c63200640438821a7ef86709ed5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.209327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1008/1995] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:34 2022] DELETE /identity/v3/users/bece5c63200640438821a7ef86709ed5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:34.216738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.217031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.217301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.244018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:34.245363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.247017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 988/1996] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:34.253111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.256086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.279373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c88cce-d457-4cb9-b27a-89b750fc91af None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:34.300805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:34.328798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1009/1997] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:34.333646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.336030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.361730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:34.362577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.362714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.362924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] PATH_INFO: `/v3/users/6734b944f3c5434985b879e5bae92fe3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.364151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6734b944f3c5434985b879e5bae92fe3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:34.367148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.405021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd90d8-33cd-41f3-a36f-565437dd2eaa None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:34.410284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:34.411238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.411995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.411995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.425302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:34.427011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.435016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6734b944f3c5434985b879e5bae92fe3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.439961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1010/1998] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:34.440946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.446115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6734b944f3c5434985b879e5bae92fe3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.447971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6734b944f3c5434985b879e5bae92fe3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.455513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aa042-6250-4477-9785-d6872d24262b service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:34.473526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:34.473526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.473526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.473526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.474078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6734b944f3c5434985b879e5bae92fe3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.490667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-554b127b-814b-49e2-b482-724f9a50c46f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6734b944f3c5434985b879e5bae92fe3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.491850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:34.493345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.506253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 989/1999] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:34 2022] DELETE /identity/v3/users/6734b944f3c5434985b879e5bae92fe3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:34.509242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1011/2000] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:34.511176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.511500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.511896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.534096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:34.535458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.612868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 990/2001] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:34.624202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.634015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:34.641034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c281-3e19-460e-a0c0-3126829446ed service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:34.678989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.681353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.696584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7687c9d-c0c6-4678-8654-7006792ad4a6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:34.704410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:34.705729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.706284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.706657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] PATH_INFO: `/v3/users/f0e01022f97340a8b8ee0a17bb451df7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.706797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc566-58d2-4982-a14f-5eb3fe0ba587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:34.707481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc566-58d2-4982-a14f-5eb3fe0ba587 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.707772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc566-58d2-4982-a14f-5eb3fe0ba587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.708027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc566-58d2-4982-a14f-5eb3fe0ba587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.708812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0e01022f97340a8b8ee0a17bb451df7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:34.711469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.719894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc566-58d2-4982-a14f-5eb3fe0ba587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:34.723721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc566-58d2-4982-a14f-5eb3fe0ba587 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:34.734631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1012/2002] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:34.777230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:34.792852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0e01022f97340a8b8ee0a17bb451df7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.804178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0e01022f97340a8b8ee0a17bb451df7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.804942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0e01022f97340a8b8ee0a17bb451df7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.826743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0e01022f97340a8b8ee0a17bb451df7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.841660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0e01022f97340a8b8ee0a17bb451df7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:34.865000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 991/2003] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:34 2022] DELETE /identity/v3/users/f0e01022f97340a8b8ee0a17bb451df7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:34.870933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:34.871242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:34.871539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:34.913700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:34.915149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:34.958678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:34.960803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.001501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1013/2004] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:35.028803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.030408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.097311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5909d7-ab14-4463-9809-4ca48f9fbb1b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.103741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.104711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.105309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.105803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] PATH_INFO: `/v3/projects/1440364a621c4923829c2981476c5d92` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.107003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1440364a621c4923829c2981476c5d92)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.109746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.131941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6864cc-6cbb-46aa-a141-cd1329f90307 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.136146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.136848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.137203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.137546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.149161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.150772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.164514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1014/2005] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:35.165565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.167791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.192298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ee1c4-f04b-4aed-88ec-b423d6c0bfa0 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.203022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1440364a621c4923829c2981476c5d92', 'request_id': 'req-ab851ded-eb99-4b51-850c-df164a34ea31'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.213221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1440364a621c4923829c2981476c5d92', 'request_id': 'req-ab851ded-eb99-4b51-850c-df164a34ea31'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.219943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1440364a621c4923829c2981476c5d92', 'request_id': 'req-ab851ded-eb99-4b51-850c-df164a34ea31'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.235082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] The token cache is being invalidate because project 1440364a621c4923829c2981476c5d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:35.235481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.236177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.243844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 992/2006] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:34 2022] DELETE /identity/v3/projects/1440364a621c4923829c2981476c5d92 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:35.245238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.246848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.304649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:35.326217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.326914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.327288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.327621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] PATH_INFO: `/v3/projects/c13b2a205af040488a364d26eae23a15` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.328484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:35.330269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13b2a205af040488a364d26eae23a15)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.332589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.380688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab851ded-eb99-4b51-850c-df164a34ea31 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.396122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.396755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.397031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.397388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] PATH_INFO: `/v3/users/9e1106d9888641afa7ae402789cbc3f2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.399033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e1106d9888641afa7ae402789cbc3f2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.401025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.437047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13b2a205af040488a364d26eae23a15', 'request_id': 'req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.449571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13b2a205af040488a364d26eae23a15', 'request_id': 'req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.482649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13b2a205af040488a364d26eae23a15', 'request_id': 'req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.518751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] The token cache is being invalidate because project c13b2a205af040488a364d26eae23a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:35.519348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.519987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.522053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1015/2007] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:35 2022] DELETE /identity/v3/projects/c13b2a205af040488a364d26eae23a15 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:35.523889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.529258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.576096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e1106d9888641afa7ae402789cbc3f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.587318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e1106d9888641afa7ae402789cbc3f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.588254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e1106d9888641afa7ae402789cbc3f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.608002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e1106d9888641afa7ae402789cbc3f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.628461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e1106d9888641afa7ae402789cbc3f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.638172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 993/2008] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:35 2022] DELETE /identity/v3/users/9e1106d9888641afa7ae402789cbc3f2 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:35.646173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.648135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.694710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95874e-c2ff-4fdb-923d-e7942c4af9b5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.779647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df35755-df61-4b61-beff-4f662452f75a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.790992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.792093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.792772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.792772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] PATH_INFO: `/v3/users/735925da3d194bc4bc7fa0c7b8c1bd5a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.794228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=735925da3d194bc4bc7fa0c7b8c1bd5a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.796757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.836234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.836885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.837161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.837439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] PATH_INFO: `/v3/projects/9220968e3b18463cbddf0123a20427f0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.838923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9220968e3b18463cbddf0123a20427f0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.842030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.859782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '735925da3d194bc4bc7fa0c7b8c1bd5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.872428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '735925da3d194bc4bc7fa0c7b8c1bd5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.873239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '735925da3d194bc4bc7fa0c7b8c1bd5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.891829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:35.908200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:35.921487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '735925da3d194bc4bc7fa0c7b8c1bd5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.925703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9220968e3b18463cbddf0123a20427f0', 'request_id': 'req-192acda3-52dd-4f77-8570-a3517ec3a6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.934996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '735925da3d194bc4bc7fa0c7b8c1bd5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.937212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9220968e3b18463cbddf0123a20427f0', 'request_id': 'req-192acda3-52dd-4f77-8570-a3517ec3a6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.942541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 994/2009] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:35 2022] DELETE /identity/v3/users/735925da3d194bc4bc7fa0c7b8c1bd5a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:35.944058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.945913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.954261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9220968e3b18463cbddf0123a20427f0', 'request_id': 'req-192acda3-52dd-4f77-8570-a3517ec3a6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.992703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] The token cache is being invalidate because project 9220968e3b18463cbddf0123a20427f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:35.993940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.994635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.997109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1016/2010] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:35 2022] DELETE /identity/v3/projects/9220968e3b18463cbddf0123a20427f0 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:36.007298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.016087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.127187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de737703-c241-4f8e-8a26-9c7e70bdf57f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.152985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.154032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.154973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.156670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] PATH_INFO: `/v3/projects/b6e34cb528274bc88d6d1a99d8e0ca11` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.159057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e34cb528274bc88d6d1a99d8e0ca11)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.178783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.210098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192acda3-52dd-4f77-8570-a3517ec3a6bd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.216657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.217362 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.218394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.219192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] PATH_INFO: `/v3/users/751ecd963f7c482799233e379d7312a1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.229517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=751ecd963f7c482799233e379d7312a1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.231616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.277024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e34cb528274bc88d6d1a99d8e0ca11', 'request_id': 'req-1cf20f88-d41c-420c-972d-a1b8a875b316'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.286937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e34cb528274bc88d6d1a99d8e0ca11', 'request_id': 'req-1cf20f88-d41c-420c-972d-a1b8a875b316'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.294339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e34cb528274bc88d6d1a99d8e0ca11', 'request_id': 'req-1cf20f88-d41c-420c-972d-a1b8a875b316'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.308559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] The token cache is being invalidate because project b6e34cb528274bc88d6d1a99d8e0ca11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:36.308948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.309639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.315648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 995/2011] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:35 2022] DELETE /identity/v3/projects/b6e34cb528274bc88d6d1a99d8e0ca11 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:36.316596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.318010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.325532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751ecd963f7c482799233e379d7312a1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.337742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751ecd963f7c482799233e379d7312a1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.369926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751ecd963f7c482799233e379d7312a1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.370667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751ecd963f7c482799233e379d7312a1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.372265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8e5e4118-4f5a-4f88-a4bf-fac974f74a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751ecd963f7c482799233e379d7312a1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.392777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1017/2012] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:36 2022] DELETE /identity/v3/users/751ecd963f7c482799233e379d7312a1 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:36.394170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.394503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.394832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.431837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:36.433200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.439552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf20f88-d41c-420c-972d-a1b8a875b316 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.467401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:36.486544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:36.545468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1018/2013] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:36.546504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.551519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.556980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.558078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.558320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.558733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] PATH_INFO: `/v3/projects/8773c5029d0f4462a8f9935d97cc361b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.560081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8773c5029d0f4462a8f9935d97cc361b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.562691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.567288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544a72-4c69-44e1-b4a1-059f0ef7c350 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.572889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.573531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.574000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.574217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] PATH_INFO: `/v3/projects/4fbe3294421f406da60509a89490c47e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.575524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fbe3294421f406da60509a89490c47e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.577945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.648322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8773c5029d0f4462a8f9935d97cc361b', 'request_id': 'req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.662154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8773c5029d0f4462a8f9935d97cc361b', 'request_id': 'req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.665099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fbe3294421f406da60509a89490c47e', 'request_id': 'req-258710d6-f61f-4fba-9c4b-e289d0bdad2c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.669592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8773c5029d0f4462a8f9935d97cc361b', 'request_id': 'req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.673968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fbe3294421f406da60509a89490c47e', 'request_id': 'req-258710d6-f61f-4fba-9c4b-e289d0bdad2c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.680490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fbe3294421f406da60509a89490c47e', 'request_id': 'req-258710d6-f61f-4fba-9c4b-e289d0bdad2c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.685353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] The token cache is being invalidate because project 8773c5029d0f4462a8f9935d97cc361b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:36.685955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.686702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6abbccb6-b49f-4f89-99ae-6e05a2b0ed09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.688633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 996/2014] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:36 2022] DELETE /identity/v3/projects/8773c5029d0f4462a8f9935d97cc361b => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:36.692541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484295b0-6ffc-416f-9980-287312691951 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.692824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484295b0-6ffc-416f-9980-287312691951 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.693100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484295b0-6ffc-416f-9980-287312691951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.697327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] The token cache is being invalidate because project 4fbe3294421f406da60509a89490c47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:36.697865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.699746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-258710d6-f61f-4fba-9c4b-e289d0bdad2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.701381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1019/2015] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:36 2022] DELETE /identity/v3/projects/4fbe3294421f406da60509a89490c47e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:36.705702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aa385-5347-408a-8c4c-e8d0609a4044 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.705702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aa385-5347-408a-8c4c-e8d0609a4044 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.705702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229aa385-5347-408a-8c4c-e8d0609a4044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.720950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-484295b0-6ffc-416f-9980-287312691951 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:36.722248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-484295b0-6ffc-416f-9980-287312691951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.731910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-229aa385-5347-408a-8c4c-e8d0609a4044 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:36.733241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-229aa385-5347-408a-8c4c-e8d0609a4044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.818946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1020/2016] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:36.823497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb179896-8693-4463-926d-9cdab42d240b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.823879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb179896-8693-4463-926d-9cdab42d240b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.824227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb179896-8693-4463-926d-9cdab42d240b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.836507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 997/2017] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:36.838573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.838991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.839290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.863498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fb179896-8693-4463-926d-9cdab42d240b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:36.864709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fb179896-8693-4463-926d-9cdab42d240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.867809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:36.869044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.873201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1021/2018] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:36.876606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb179896-8693-4463-926d-9cdab42d240b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.884264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb179896-8693-4463-926d-9cdab42d240b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.888315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 998/2019] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:36.888315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.888315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.897541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.898423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.898781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.899021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.899724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:36.900027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.901094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e18bc-2a6b-4b17-b01f-20a4227d8adb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.901786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8fdc8-7084-4a36-bab0-4d7597cbb642 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.905975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.906623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.906908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.907415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] PATH_INFO: `/v3/projects/805be4f47d064d81a769c8ee3a61c4fd` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.909157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805be4f47d064d81a769c8ee3a61c4fd)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.911093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.916741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1022/2020] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:36.918680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.920104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.920590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.949467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:36.950840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.961431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1023/2021] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:36.962615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.965917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:36.975302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577af85-5f6c-4e08-98b2-97f54d4c20b5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.978992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805be4f47d064d81a769c8ee3a61c4fd', 'request_id': 'req-d19c030d-c077-4155-9b9a-c82814627590'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.991317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805be4f47d064d81a769c8ee3a61c4fd', 'request_id': 'req-d19c030d-c077-4155-9b9a-c82814627590'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.999631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805be4f47d064d81a769c8ee3a61c4fd', 'request_id': 'req-d19c030d-c077-4155-9b9a-c82814627590'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.011436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] The token cache is being invalidate because project 805be4f47d064d81a769c8ee3a61c4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:37.011886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.012702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.012845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.014397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 999/2022] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:36 2022] DELETE /identity/v3/projects/805be4f47d064d81a769c8ee3a61c4fd => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.016023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.018201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.046413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.047082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.047402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.047678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.048347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:37.048662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.050504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.065591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1024/2023] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:37.066969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.069288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.084606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51e389-8d33-4bf8-904f-a25546815b82 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.090293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.091131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.091447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.091795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.092443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:37.092860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.094213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c030d-c077-4155-9b9a-c82814627590 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.094863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.099368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.099995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.100281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.100554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.101227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.103151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.108726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1025/2024] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:37.109619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.111321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.124502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f75abe-a8cf-4961-a6fd-e7bd6a5d36d2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.130388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.131045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.131391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.131926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] PATH_INFO: `/v3/users/ec5db64a317842d993ff99ebbd711969` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.132908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5db64a317842d993ff99ebbd711969)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.135529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.138202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1000/2025] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:37.138963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.141235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.196012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5db64a317842d993ff99ebbd711969'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.205578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5db64a317842d993ff99ebbd711969'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.220912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5db64a317842d993ff99ebbd711969'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.221511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5db64a317842d993ff99ebbd711969'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.222129 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5db64a317842d993ff99ebbd711969'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.224545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa175710-8c16-4cd4-90be-2cc43a68bd6d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.229382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.230066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.230417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.230778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.231516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.234358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.242441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1026/2026] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:37 2022] DELETE /identity/v3/users/ec5db64a317842d993ff99ebbd711969 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.242441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.247881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.276203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1001/2027] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:37.277367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.280464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.331190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.360442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6350-ed10-43ff-8820-cfdec565116d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.367075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.368179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.368639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.369097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.369763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.371857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.377440 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b08a54-e3f3-43af-9fe9-9c23c68e6694 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.383258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.383969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.384342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.384683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.385655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.387922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.413937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1027/2028] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/projects => generated 401 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:37.415062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.417627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.418329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1002/2029] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:37.419312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.433255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cc837-1197-4ced-91c7-b20fc0d2b1e0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.443571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.504976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816ce98-0c71-4225-936d-b6062817ca64 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.510708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.511454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.511740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.512015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.512652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.514651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.520578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.521594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.521594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.521802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.522554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.524595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.539412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1028/2030] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:37.540249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.551676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1003/2031] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/users => generated 301 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:37.552526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.556175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a146f-55db-405f-b938-9a0ed7656829 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.562031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.562549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.562889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.563291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.563998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.564954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb59152-aa01-45db-b96f-082302c55bf6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.566155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.572205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.573175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.574020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.574371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.575074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.576880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.579574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1029/2032] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:37.582651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.595243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1004/2033] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:37.595415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab883e-ff6f-4429-a70c-08a7a89c71eb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.595527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.604881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.605701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.606320 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.607116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.607871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd81b38-3d8a-43f2-a877-52b18a8d245e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.608456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.610711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.614403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.615095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.615479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.615871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.616593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.618706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.623178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1030/2034] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:37.624141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.627721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.636903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1005/2035] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:37.638003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.645944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934abc6f-6f3e-4386-8769-b7ae991c1ef1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.649649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa61b1-8d31-4077-80f7-e00d4c5b933a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.651112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.652121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.652471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.652852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] PATH_INFO: `/v3/projects/2ad670ff862940439fe89608996e6660` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.653964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ad670ff862940439fe89608996e6660)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.654618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.655817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.656107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.656385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.656905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.657017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.658815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.670106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1006/2036] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:37.670872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.684976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c4324-9812-4669-90e7-55dccc6e5f12 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.691180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.692028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.692305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.692602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.693202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.694787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.706594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1007/2037] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:37.707366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.716608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ad670ff862940439fe89608996e6660', 'request_id': 'req-7a36d688-f5ab-497f-a77a-8b654f125f30'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.717171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf03b2-c28f-4750-b7f7-e03e7cde319d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.725457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ad670ff862940439fe89608996e6660', 'request_id': 'req-7a36d688-f5ab-497f-a77a-8b654f125f30'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.731565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ad670ff862940439fe89608996e6660', 'request_id': 'req-7a36d688-f5ab-497f-a77a-8b654f125f30'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.743859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] The token cache is being invalidate because project 2ad670ff862940439fe89608996e6660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:37.744142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.744822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.746315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1031/2038] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:37 2022] DELETE /identity/v3/projects/2ad670ff862940439fe89608996e6660 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.747111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.748490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.788855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.789632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.790074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.790373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] PATH_INFO: `/v3/projects/186a1113c2cf400ebae35d9c53716df0/users/a2e6cb9989eb44b792d19eb5b7ed5e5a/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.806667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=186a1113c2cf400ebae35d9c53716df0, user_id=a2e6cb9989eb44b792d19eb5b7ed5e5a, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.809368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.811680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.824601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1008/2039] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:37 2022] PUT /identity/v3/projects/186a1113c2cf400ebae35d9c53716df0/users/a2e6cb9989eb44b792d19eb5b7ed5e5a/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.825723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36d688-f5ab-497f-a77a-8b654f125f30 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.828205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.832741 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.834525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.909447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f3a4f-fdf8-4370-8c50-b50b44bbb3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.910094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f3a4f-fdf8-4370-8c50-b50b44bbb3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.910382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f3a4f-fdf8-4370-8c50-b50b44bbb3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.910656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f3a4f-fdf8-4370-8c50-b50b44bbb3b6 admin admin] PATH_INFO: `/v3/projects/427cd367af054271826f33b9cbead003/users/9cd5036364ac4ced94c21f3a48ebfc80/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.934158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dae8ab-271d-4515-aaab-56a37f95cfa6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.940068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d77eb-e020-4332-be41-90969b586868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.940748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d77eb-e020-4332-be41-90969b586868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.941075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d77eb-e020-4332-be41-90969b586868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.941390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d77eb-e020-4332-be41-90969b586868 admin admin] PATH_INFO: `/v3/projects/8c4344fe2ae142fca7d3b983481d9fb9/users/a81a35a4b05b4cf198445c9cd8925f92/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.946290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f3a4f-fdf8-4370-8c50-b50b44bbb3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427cd367af054271826f33b9cbead003, user_id=9cd5036364ac4ced94c21f3a48ebfc80, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.951353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f3a4f-fdf8-4370-8c50-b50b44bbb3b6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.973881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d77eb-e020-4332-be41-90969b586868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4344fe2ae142fca7d3b983481d9fb9, user_id=a81a35a4b05b4cf198445c9cd8925f92, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.976630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d77eb-e020-4332-be41-90969b586868 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.990947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1032/2040] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:37 2022] PUT /identity/v3/projects/427cd367af054271826f33b9cbead003/users/9cd5036364ac4ced94c21f3a48ebfc80/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.990947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e80c0-9584-4690-af7b-ca34605f7481 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.990947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e80c0-9584-4690-af7b-ca34605f7481 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.990947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e80c0-9584-4690-af7b-ca34605f7481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.000836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1009/2041] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:37 2022] PUT /identity/v3/projects/8c4344fe2ae142fca7d3b983481d9fb9/users/a81a35a4b05b4cf198445c9cd8925f92/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:38.007598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a2de2-bb74-4aa8-99f2-65652ca8cdc3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.007893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a2de2-bb74-4aa8-99f2-65652ca8cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.008153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a2de2-bb74-4aa8-99f2-65652ca8cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.029024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-eb3e80c0-9584-4690-af7b-ca34605f7481 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.030881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eb3e80c0-9584-4690-af7b-ca34605f7481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.055528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ce6a2de2-bb74-4aa8-99f2-65652ca8cdc3 None None] MFA Rules not processed for user `a2e6cb9989eb44b792d19eb5b7ed5e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.057503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce6a2de2-bb74-4aa8-99f2-65652ca8cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.119465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1033/2042] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.120747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bc0f0-c590-499a-863b-d29981be8bbd None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.121039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bc0f0-c590-499a-863b-d29981be8bbd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.121315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bc0f0-c590-499a-863b-d29981be8bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.152523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-486bc0f0-c590-499a-863b-d29981be8bbd None None] MFA Rules not processed for user `9cd5036364ac4ced94c21f3a48ebfc80`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.153761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-486bc0f0-c590-499a-863b-d29981be8bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.167018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1010/2043] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.168447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.168816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.169304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.199934 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] MFA Rules not processed for user `a81a35a4b05b4cf198445c9cd8925f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.201343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.247331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1034/2044] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.247331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.247331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.247331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.278866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.280164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.283840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.288670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1035/2045] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.289465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.290707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.297454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1011/2046] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.298518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.299748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.310341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.352849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc756bde-3d08-4540-9fb0-51a394514ce0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.356335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd624-2aeb-40fa-b2d8-6bae5451e255 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.356534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.357152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.357432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.357703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.360105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.360700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.360974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.361239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.366224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.367716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.369862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.371313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.377374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1036/2047] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:38.378176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.380985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1012/2048] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:38.381651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.388961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128529f-0ebb-429e-9646-747c40a445b7 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.394809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.403754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604205d-afd5-4e75-a6d3-fd3150bfe717 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.403966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.403966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.403966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.403966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:38.403966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.403966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.404353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.404353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.404353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.404353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.404621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.404621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.417317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1037/2049] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:38.418126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.427955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c4ce3-bf46-4208-98a8-902c3a811192 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.435128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.436187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.437213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.437741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] PATH_INFO: `/v3/users/9cd5036364ac4ced94c21f3a48ebfc80` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.439651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd5036364ac4ced94c21f3a48ebfc80)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.444384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1013/2050] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:38.444384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.444580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.444653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.505521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd5036364ac4ced94c21f3a48ebfc80'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.512798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c893163-ea3a-4879-b2f1-ad35df7c09ab admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.513493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd5036364ac4ced94c21f3a48ebfc80'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.517745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.518475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.518651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.518919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] PATH_INFO: `/v3/users/a2e6cb9989eb44b792d19eb5b7ed5e5a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.520136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e6cb9989eb44b792d19eb5b7ed5e5a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.522007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.527127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd5036364ac4ced94c21f3a48ebfc80'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.527870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd5036364ac4ced94c21f3a48ebfc80'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.528523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd5036364ac4ced94c21f3a48ebfc80'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.541683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1038/2051] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:38 2022] DELETE /identity/v3/users/9cd5036364ac4ced94c21f3a48ebfc80 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:38.542760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.544125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.573116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e6cb9989eb44b792d19eb5b7ed5e5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.580659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e6cb9989eb44b792d19eb5b7ed5e5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.581855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e6cb9989eb44b792d19eb5b7ed5e5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.595426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e6cb9989eb44b792d19eb5b7ed5e5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.600703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.603048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e6cb9989eb44b792d19eb5b7ed5e5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.611051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485d704-5714-41f4-810a-3556207beee4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.611787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1014/2052] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:38 2022] DELETE /identity/v3/users/a2e6cb9989eb44b792d19eb5b7ed5e5a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:38.615726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.617570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.631610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.669302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04617c-604b-4809-931c-a7ed2164b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.669916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04617c-604b-4809-931c-a7ed2164b5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.670192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04617c-604b-4809-931c-a7ed2164b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.670468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04617c-604b-4809-931c-a7ed2164b5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.671132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04617c-604b-4809-931c-a7ed2164b5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.673179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04617c-604b-4809-931c-a7ed2164b5ac admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.686294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c3bbe-c031-4ccc-839b-ca8340ef8ead admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.691424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72981c-0c58-425f-9a92-87d08bd43be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.692130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72981c-0c58-425f-9a92-87d08bd43be3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.692412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72981c-0c58-425f-9a92-87d08bd43be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.692695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72981c-0c58-425f-9a92-87d08bd43be3 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.693447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72981c-0c58-425f-9a92-87d08bd43be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.695472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72981c-0c58-425f-9a92-87d08bd43be3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.700434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1039/2053] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:38.701788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.702590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.702869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.720536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1015/2054] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/users => generated 306 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:38.720536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.720536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.720536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.722252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.723469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.738446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.739790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.784806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1040/2055] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.785635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.786880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.799139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e292c1-fd09-43fc-aa94-792af87aa178 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.804781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.805392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.805673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.805946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.806582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.807455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1016/2056] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.808556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.808944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.828105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bc9a7-e6b7-4415-ad6c-2513ca90658c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.831932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1041/2057] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/users => generated 286 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:38.832978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.833256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.833595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.833872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.834139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.835063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.835177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.836728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.846840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1017/2058] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:38.850920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.852652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.894075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4e147-9936-4f45-90d6-152ac6727a9a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.897934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.898551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.898982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.899308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.910804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.912311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.913076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2542e-f18e-4ded-a9b9-79db9c628c66 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.916959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.917599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.917886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.918251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.924883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1042/2059] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:38.925905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.928550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.929988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.937512 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b20f3b-21d3-43e2-bfaa-049c0f3dd8a2 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.940832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1018/2060] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:38.941496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.942659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.943417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.943697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.943973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.944613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.946238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.953477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b989677-4cd5-4be7-ba20-c647de9f45e3 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.957898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1043/2061] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:38.961360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.961689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.962193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.962632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.962756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] PATH_INFO: `/v3/projects/df8e51b05f734870a085a11c1fe23dc1/users/fb609f936c454427a39bcfaacbdbdaad/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.973683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b0ed2-e294-42c9-ab4e-d0871b4a8ba1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.980345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.981222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.981533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.981804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.982451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.984642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.987623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8e51b05f734870a085a11c1fe23dc1, user_id=fb609f936c454427a39bcfaacbdbdaad, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.992367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.996157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.998618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1044/2062] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:39.004480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.015253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1019/2063] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:38 2022] PUT /identity/v3/projects/df8e51b05f734870a085a11c1fe23dc1/users/fb609f936c454427a39bcfaacbdbdaad/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:39.020126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.024677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94526f4-751e-4f8c-8792-82e5c0105608 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.042465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:39.126907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10836911-0f76-4f73-b838-245e33cdc7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.127912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10836911-0f76-4f73-b838-245e33cdc7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.128262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10836911-0f76-4f73-b838-245e33cdc7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.128603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10836911-0f76-4f73-b838-245e33cdc7d8 admin admin] PATH_INFO: `/v3/projects/4cdc754836b5483481ce53ad9d8843f8/users/791a3dc3f4b8427387d6c6c0b8fd549f/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.134537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81d4cb-ba66-4b40-a127-fffbd8dd00e6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.141803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.142620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.143715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.144001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.144653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.146882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.148541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10836911-0f76-4f73-b838-245e33cdc7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdc754836b5483481ce53ad9d8843f8, user_id=791a3dc3f4b8427387d6c6c0b8fd549f, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.152037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10836911-0f76-4f73-b838-245e33cdc7d8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.163862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1020/2064] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:39.164545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.175132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1045/2065] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:39 2022] PUT /identity/v3/projects/4cdc754836b5483481ce53ad9d8843f8/users/791a3dc3f4b8427387d6c6c0b8fd549f/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:39.189719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.189949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.190820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.200568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45c8d-e212-4130-a2e4-9729c0d0c650 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.272962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] MFA Rules not processed for user `791a3dc3f4b8427387d6c6c0b8fd549f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:39.277376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.417693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20323504-ff40-4139-b71b-a0515576025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.418453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323504-ff40-4139-b71b-a0515576025a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.418737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323504-ff40-4139-b71b-a0515576025a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.419013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323504-ff40-4139-b71b-a0515576025a admin admin] PATH_INFO: `/v3/projects/df8e51b05f734870a085a11c1fe23dc1/users/fb609f936c454427a39bcfaacbdbdaad/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.424337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323504-ff40-4139-b71b-a0515576025a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8e51b05f734870a085a11c1fe23dc1, user_id=fb609f936c454427a39bcfaacbdbdaad, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.428615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323504-ff40-4139-b71b-a0515576025a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.438113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1046/2066] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:39.451047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.453029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.486395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1021/2067] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:39 2022] PUT /identity/v3/projects/df8e51b05f734870a085a11c1fe23dc1/users/fb609f936c454427a39bcfaacbdbdaad/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:39.503129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.503560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.503941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.564986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] MFA Rules not processed for user `fb609f936c454427a39bcfaacbdbdaad`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:39.566939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.635243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf6945-96bc-43c5-b62c-5fc3082b7eec None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.675317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:39.713203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1022/2068] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:39.713203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.715518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.728485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:39.728776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.729377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.730825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.730825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.814861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150dac4-42de-4db1-aacb-a648bbe42046 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.820955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.821799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.822504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.822955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] PATH_INFO: `/v3/projects/427cd367af054271826f33b9cbead003` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.824414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=427cd367af054271826f33b9cbead003)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.825617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.827758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.827901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.844683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1047/2069] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:39.849065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.853695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.895302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66f4c2-64a2-4f61-9201-e4205221f50a service glance] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.902382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.903469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.904267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.904739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] PATH_INFO: `/v3/projects/186a1113c2cf400ebae35d9c53716df0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.906422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=186a1113c2cf400ebae35d9c53716df0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.909176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.929885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427cd367af054271826f33b9cbead003', 'request_id': 'req-b54be189-d9e8-4e30-9540-d0dedaec1fcb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.941090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427cd367af054271826f33b9cbead003', 'request_id': 'req-b54be189-d9e8-4e30-9540-d0dedaec1fcb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.948562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427cd367af054271826f33b9cbead003', 'request_id': 'req-b54be189-d9e8-4e30-9540-d0dedaec1fcb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.964662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] The token cache is being invalidate because project 427cd367af054271826f33b9cbead003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:39.965227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.965995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.968012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1023/2070] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:39 2022] DELETE /identity/v3/projects/427cd367af054271826f33b9cbead003 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:39.968922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.970389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.983954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186a1113c2cf400ebae35d9c53716df0', 'request_id': 'req-a5358b00-316e-404b-969c-765efb8436f2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.994711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186a1113c2cf400ebae35d9c53716df0', 'request_id': 'req-a5358b00-316e-404b-969c-765efb8436f2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.008590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186a1113c2cf400ebae35d9c53716df0', 'request_id': 'req-a5358b00-316e-404b-969c-765efb8436f2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.033580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] The token cache is being invalidate because project 186a1113c2cf400ebae35d9c53716df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:40.033745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.035264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5358b00-316e-404b-969c-765efb8436f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.039187 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1048/2071] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:39 2022] DELETE /identity/v3/projects/186a1113c2cf400ebae35d9c53716df0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:40.040286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597216f-550e-4d0e-9449-b13b0899485f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.040572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597216f-550e-4d0e-9449-b13b0899485f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.040851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597216f-550e-4d0e-9449-b13b0899485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.076148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6597216f-550e-4d0e-9449-b13b0899485f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:40.078101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6597216f-550e-4d0e-9449-b13b0899485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.101781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54be189-d9e8-4e30-9540-d0dedaec1fcb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.222607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:40.225094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.226656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.226954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.227399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.230283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.245547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1049/2072] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:40.255632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.256004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.256709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.262404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:40.286492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:40.288926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.301103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1050/2073] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:40.302677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.314770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba7822-33da-409e-8ba9-b2e152e8204a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.319727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.320269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.320540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.320810 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.321509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:40.321809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.323469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.336666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1051/2074] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:40.341409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.352297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c440ffb-8f2b-402a-97af-4238206ded67 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.357551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.358873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.359359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.359705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.360708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.363164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.370293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.372158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.384292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1024/2075] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:40.399608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1052/2076] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:40.402655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.413668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f3700-9071-4a9d-a337-637291291acc service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.516580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.517172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.517288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.517545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.517961 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.519113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.520297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.521233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.563842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1025/2077] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:40.567715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.581116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc2eb-b4b9-4429-8f5b-5e3b3e86eb17 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.586480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.587314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.587600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.587875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.589203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.590399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.618132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1026/2078] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:40.619364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b612b-258f-4197-8bb0-0a12bd125a96 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.620154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.623415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.624103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.624344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.624709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.626420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.636415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6656dea-6906-47ad-be7f-769cd6e8122d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.649568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.653075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.653880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.655853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.657888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.661025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.684926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1027/2079] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:40.689269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.708284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c5371-1b79-41e4-8f06-47669f3405c1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.711954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.715194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff589d-5ac2-477e-aa45-81f112c5c20a service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.722733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.723604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.726491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.726782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] PATH_INFO: `/v3/projects/05a7808809fe4d94931f3f8afcd03b7e/users/61f38a09d0f7447694f38efa8fbc2934/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.730016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1053/2080] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:40.748427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a7808809fe4d94931f3f8afcd03b7e, user_id=61f38a09d0f7447694f38efa8fbc2934, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.753970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.787020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1028/2081] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:40 2022] PUT /identity/v3/projects/05a7808809fe4d94931f3f8afcd03b7e/users/61f38a09d0f7447694f38efa8fbc2934/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:40.790370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.791154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.791497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.824145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] MFA Rules not processed for user `61f38a09d0f7447694f38efa8fbc2934`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:40.825304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.892718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1054/2082] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:40.906126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.908339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.964377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:40.969324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66dfea-7593-4211-8d22-8503817108bc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.974058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.974848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.975344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.975621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.985007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.986484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.995900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1029/2083] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:50.888886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:50.991599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:53.378888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:53.381293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.445463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:53.446953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.451454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3079c-366b-43c7-9ef4-048c6f1852d9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.455876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.456502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.456776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.457045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] PATH_INFO: `/v3/users/8b6ffa903cf64bd6b78648dad1f53839` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.458151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6ffa903cf64bd6b78648dad1f53839)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.460093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.460886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282168a-640e-4a03-8a95-ea8706c1ea40 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.465486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.466105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.466388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.466683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] PATH_INFO: `/v3/users/a81a35a4b05b4cf198445c9cd8925f92` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.467836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81a35a4b05b4cf198445c9cd8925f92)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.470336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.517254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6ffa903cf64bd6b78648dad1f53839'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.524960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81a35a4b05b4cf198445c9cd8925f92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.527150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6ffa903cf64bd6b78648dad1f53839'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.532824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81a35a4b05b4cf198445c9cd8925f92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.533507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81a35a4b05b4cf198445c9cd8925f92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.545601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6ffa903cf64bd6b78648dad1f53839'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.547463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6ffa903cf64bd6b78648dad1f53839'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.547584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81a35a4b05b4cf198445c9cd8925f92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.548085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d8d523a-6ad7-470b-bdd5-022b85beb0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6ffa903cf64bd6b78648dad1f53839'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.556633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f6614204-90f3-4be9-9a7d-89621065dab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81a35a4b05b4cf198445c9cd8925f92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.564766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1030/2084] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:53 2022] DELETE /identity/v3/users/a81a35a4b05b4cf198445c9cd8925f92 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:53.565138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1055/2085] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:53 2022] DELETE /identity/v3/users/8b6ffa903cf64bd6b78648dad1f53839 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:53.579449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.579557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.579647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.579731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.579814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.579905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.597382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:53.598603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.600223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:53.601419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.666795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1031/2086] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:53.672938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1056/2087] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:53.677432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:53.679696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.682934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:53.684319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.706411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:53.767588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93e432-3a7e-483c-ad13-39e0b99b416b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.770192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943933d-4282-4be6-acc9-0d85a8a16b4a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.772079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.772725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.772966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.773240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.773983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.774712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.774987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.775299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.786137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.787910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.788556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.790039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.801446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1032/2088] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:53.806795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1057/2089] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:54.136314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:54.138425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:54.171896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37c60b-428d-46f2-9612-82b824621fce service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:54.181429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:54.182441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:54.182969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:54.183521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] PATH_INFO: `/v3/projects/8c4344fe2ae142fca7d3b983481d9fb9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:54.185386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4344fe2ae142fca7d3b983481d9fb9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:54.187929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:54.207759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:54.254585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4344fe2ae142fca7d3b983481d9fb9', 'request_id': 'req-699d07d6-ea76-459b-90bb-19cc0e1fa27e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.264299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4344fe2ae142fca7d3b983481d9fb9', 'request_id': 'req-699d07d6-ea76-459b-90bb-19cc0e1fa27e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.272618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4344fe2ae142fca7d3b983481d9fb9', 'request_id': 'req-699d07d6-ea76-459b-90bb-19cc0e1fa27e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.288841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] The token cache is being invalidate because project 8c4344fe2ae142fca7d3b983481d9fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:54.289328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.290145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.292131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1033/2090] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:54 2022] DELETE /identity/v3/projects/8c4344fe2ae142fca7d3b983481d9fb9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:54.296508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:54.308300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4b6ff-df16-4d01-a11c-5b966c5bee6e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:54.351354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:54.353149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:54.407101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:54.407912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:54.408260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:54.408599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] PATH_INFO: `/v3/users/fb609f936c454427a39bcfaacbdbdaad` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:54.410058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb609f936c454427a39bcfaacbdbdaad)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:54.412391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:54.464817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d07d6-ea76-459b-90bb-19cc0e1fa27e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:54.478783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:54.479874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:54.480563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:54.480885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] PATH_INFO: `/v3/projects/8194ac89d7394e4f9193411d8bce66fb` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:54.482102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8194ac89d7394e4f9193411d8bce66fb)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:54.485151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:54.511024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb609f936c454427a39bcfaacbdbdaad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.515508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb609f936c454427a39bcfaacbdbdaad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.535831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb609f936c454427a39bcfaacbdbdaad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.537124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb609f936c454427a39bcfaacbdbdaad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.537822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb609f936c454427a39bcfaacbdbdaad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.552954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8194ac89d7394e4f9193411d8bce66fb', 'request_id': 'req-b517b9af-3821-4458-a444-7e829774dbe7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.555948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1058/2091] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:54 2022] DELETE /identity/v3/users/fb609f936c454427a39bcfaacbdbdaad => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:54.568558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8194ac89d7394e4f9193411d8bce66fb', 'request_id': 'req-b517b9af-3821-4458-a444-7e829774dbe7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.574922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8194ac89d7394e4f9193411d8bce66fb', 'request_id': 'req-b517b9af-3821-4458-a444-7e829774dbe7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.590609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] The token cache is being invalidate because project 8194ac89d7394e4f9193411d8bce66fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:54.590914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.591700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.593595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1034/2092] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:54 2022] DELETE /identity/v3/projects/8194ac89d7394e4f9193411d8bce66fb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:54.597656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:54.599339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:54.686261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:54.715828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c3f09-5e88-4d1b-89d2-0c8f69d99020 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:54.722336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:54.723118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:54.723528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:54.723879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] PATH_INFO: `/v3/users/e63a8394bd82448ca84ef78ee92362f3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:54.725318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63a8394bd82448ca84ef78ee92362f3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:54.727487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:54.815931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63a8394bd82448ca84ef78ee92362f3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.825303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63a8394bd82448ca84ef78ee92362f3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.842106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63a8394bd82448ca84ef78ee92362f3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.843336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63a8394bd82448ca84ef78ee92362f3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.844329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63a8394bd82448ca84ef78ee92362f3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:54.862508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1059/2093] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:54 2022] DELETE /identity/v3/users/e63a8394bd82448ca84ef78ee92362f3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:54.906464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:54.908676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:54.979302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:54.981140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:54.982934 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:55.024913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b9af-3821-4458-a444-7e829774dbe7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.099835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.100075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.100297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.100569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] PATH_INFO: `/v3/projects/df8e51b05f734870a085a11c1fe23dc1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.101726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8e51b05f734870a085a11c1fe23dc1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.104439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.113887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbf889-8be1-487e-a4fe-1c56190e3933 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.120342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.121211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.121812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.122179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] PATH_INFO: `/v3/users/61f38a09d0f7447694f38efa8fbc2934` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.123581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f38a09d0f7447694f38efa8fbc2934)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.126269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.162121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8e51b05f734870a085a11c1fe23dc1', 'request_id': 'req-95391a18-00f4-44a9-9a9c-6314318f283a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.173436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8e51b05f734870a085a11c1fe23dc1', 'request_id': 'req-95391a18-00f4-44a9-9a9c-6314318f283a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.183742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8e51b05f734870a085a11c1fe23dc1', 'request_id': 'req-95391a18-00f4-44a9-9a9c-6314318f283a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.195544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f38a09d0f7447694f38efa8fbc2934'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.204567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] The token cache is being invalidate because project df8e51b05f734870a085a11c1fe23dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:55.204971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.205857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95391a18-00f4-44a9-9a9c-6314318f283a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.207960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f38a09d0f7447694f38efa8fbc2934'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.208363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1035/2094] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:54 2022] DELETE /identity/v3/projects/df8e51b05f734870a085a11c1fe23dc1 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:55.213184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9845e-e3b2-43ae-b336-72c6c8936d47 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.213473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9845e-e3b2-43ae-b336-72c6c8936d47 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.213757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9845e-e3b2-43ae-b336-72c6c8936d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.233324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:55.245726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f38a09d0f7447694f38efa8fbc2934'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.246397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f38a09d0f7447694f38efa8fbc2934'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.247075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f38a09d0f7447694f38efa8fbc2934'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.250203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-48f9845e-e3b2-43ae-b336-72c6c8936d47 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:55.251484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-48f9845e-e3b2-43ae-b336-72c6c8936d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.265381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1060/2095] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:54 2022] DELETE /identity/v3/users/61f38a09d0f7447694f38efa8fbc2934 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:55.266763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.268292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.318487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1036/2096] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:55.320208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d62a-8170-425b-988b-476740b614a1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.320698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d62a-8170-425b-988b-476740b614a1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.321193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d62a-8170-425b-988b-476740b614a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.336566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:55.343526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b6a8d62a-8170-425b-988b-476740b614a1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:55.345810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b6a8d62a-8170-425b-988b-476740b614a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.356006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84afa02-74b8-4d8e-9ccd-249428dfe4b6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.361499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1037/2097] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:55.361854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.362669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.363016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.363396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] PATH_INFO: `/v3/projects/12d5a7195d9f4a468e45982332697627` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.364720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d5a7195d9f4a468e45982332697627)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.365352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.365502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.365771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.367060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.393561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:55.395089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.408796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1038/2098] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:55.408796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.410406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.419774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d5a7195d9f4a468e45982332697627', 'request_id': 'req-fb26672c-80c2-49c7-b239-ec7ec7696ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.428882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d5a7195d9f4a468e45982332697627', 'request_id': 'req-fb26672c-80c2-49c7-b239-ec7ec7696ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.435900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d5a7195d9f4a468e45982332697627', 'request_id': 'req-fb26672c-80c2-49c7-b239-ec7ec7696ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.448824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] The token cache is being invalidate because project 12d5a7195d9f4a468e45982332697627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:55.449150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.449959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:55.455388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1061/2099] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:55 2022] DELETE /identity/v3/projects/12d5a7195d9f4a468e45982332697627 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:55.455388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.455388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.485279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f015dc6-49cd-4144-8b5b-bf3bfaecee0e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.557124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.557752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.558220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.558417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.560258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.572856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.573214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26672c-80c2-49c7-b239-ec7ec7696ed1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.574305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878b2a4-2a36-4c03-92e1-49d9a809c0ed service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.579185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.580057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.580343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.580614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.581284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:55.581592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.583280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.586913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1039/2100] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:55.588439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37724-04b9-4813-8fec-df30cf57b945 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.588901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37724-04b9-4813-8fec-df30cf57b945 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.589498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f37724-04b9-4813-8fec-df30cf57b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.597938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1062/2101] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:55.602317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.610027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:55.614377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d3f37724-04b9-4813-8fec-df30cf57b945 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:55.615685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d3f37724-04b9-4813-8fec-df30cf57b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.616367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcbc29-c2a8-4525-8ac2-40f15c948807 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.625926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1040/2102] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:55.635572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.636057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.636681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.641279 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.641994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.642348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.642689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.643811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.646014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.668365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:55.670786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:55.683793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1041/2103] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:55.690109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.692132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1063/2104] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/projects => generated 367 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:55.697984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.702526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7613-854c-4d13-8003-edd57552ca1f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.798559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.799299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.799591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.799909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.800613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:55.801045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.802823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.819081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1042/2105] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:55.827643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.854665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080f3fb-62a4-457d-8e17-90884602e29a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.863452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b5d90-28f3-47c6-88fc-f5395bd9e786 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.863738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.864480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.864869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.865233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.866400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.869002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.870517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:55.871927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:55.872377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:55.872718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:55.873371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:55.875893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:55.908406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1064/2106] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:55.914668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1043/2107] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:55.918426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.925809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:55.931758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297ee5-6eb5-43e7-bb1b-79f5a7da3afe admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:55.960110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:56.070463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.071180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.071582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.072074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.072732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.074404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.078116 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8904553-57ee-4bf4-9bc6-a16eb69dc6ec admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.084925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.085634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.085993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.089532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.089688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1065/2108] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:56.089688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.089858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.089964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.090254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.103280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6118c6-afde-4c3a-b47c-0dcd7376ecd5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.111431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.112319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.116413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.116413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] PATH_INFO: `/v3/projects/05a7808809fe4d94931f3f8afcd03b7e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.118812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a7808809fe4d94931f3f8afcd03b7e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.121919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.127409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1044/2109] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:55 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:56.127409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.138702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15b25c-0500-4fb4-83d9-e83571a5e966 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.143275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.144014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.145051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.145051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.145766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.147507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.163734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1045/2110] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:56.164846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.182974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959d07-3f34-4d82-a5f4-ae2fe6d1e0e8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.201493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a7808809fe4d94931f3f8afcd03b7e', 'request_id': 'req-15707609-0df3-4912-be04-91771b54eab3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:56.214268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a7808809fe4d94931f3f8afcd03b7e', 'request_id': 'req-15707609-0df3-4912-be04-91771b54eab3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:56.221599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:56.225346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a7808809fe4d94931f3f8afcd03b7e', 'request_id': 'req-15707609-0df3-4912-be04-91771b54eab3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:56.249351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] The token cache is being invalidate because project 05a7808809fe4d94931f3f8afcd03b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:56.249545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:56.250281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:56.255087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1066/2111] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:56 2022] DELETE /identity/v3/projects/05a7808809fe4d94931f3f8afcd03b7e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:56.255087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.256843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.263282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266b94-0738-4109-9178-4af3e31d54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.263990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266b94-0738-4109-9178-4af3e31d54f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.264331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266b94-0738-4109-9178-4af3e31d54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.264672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266b94-0738-4109-9178-4af3e31d54f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.265956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72266b94-0738-4109-9178-4af3e31d54f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.268246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72266b94-0738-4109-9178-4af3e31d54f9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.276886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707609-0df3-4912-be04-91771b54eab3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.283925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1046/2112] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:56.284104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd755061-40c7-4679-84db-13ff069e373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.284834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755061-40c7-4679-84db-13ff069e373c admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.285125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755061-40c7-4679-84db-13ff069e373c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.285404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755061-40c7-4679-84db-13ff069e373c admin admin] PATH_INFO: `/v3/projects/6e75447b10894f0c92260947a00a060d/users/39ce0b6582a544d893c857e36e2eecf2/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.285534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.285890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.286255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.310417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:56.310649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd755061-40c7-4679-84db-13ff069e373c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e75447b10894f0c92260947a00a060d, user_id=39ce0b6582a544d893c857e36e2eecf2, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.311789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.314078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd755061-40c7-4679-84db-13ff069e373c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.334817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1047/2113] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:56.334817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.338127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.343235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1067/2114] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:56 2022] PUT /identity/v3/projects/6e75447b10894f0c92260947a00a060d/users/39ce0b6582a544d893c857e36e2eecf2/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:56.347123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b5777-7213-422b-a3c9-3a23210240d4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.347490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b5777-7213-422b-a3c9-3a23210240d4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.347922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b5777-7213-422b-a3c9-3a23210240d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.356327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40219be-34c1-46b7-90a3-4baeea885906 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.381304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-080b5777-7213-422b-a3c9-3a23210240d4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:56.383012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-080b5777-7213-422b-a3c9-3a23210240d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.452292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.453498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.453787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.454177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.454858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.456593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.468065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1068/2115] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:56.468065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e31431-46f6-469c-a519-eb0b4d924cc0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.468065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e31431-46f6-469c-a519-eb0b4d924cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.468065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e31431-46f6-469c-a519-eb0b4d924cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.474511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1048/2116] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:56.475837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.487880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c3dbc-37c6-4045-a025-02c3fe6aaa4f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.493715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.495861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.496326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.496614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.497423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:56.497738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.499418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.516305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1049/2117] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:56.516305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.523308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b3e31431-46f6-469c-a519-eb0b4d924cc0 None None] MFA Rules not processed for user `39ce0b6582a544d893c857e36e2eecf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:56.524637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b3e31431-46f6-469c-a519-eb0b4d924cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.524778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:56.532368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c55b80-a990-4d64-b1ce-521976c7ab97 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.539483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.540578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.541092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.541646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] PATH_INFO: `/v3/projects/3650aec309f5416599fe3bf7627d06f5/users/8888c1586b224814b92ea780b9149994/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.564031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3650aec309f5416599fe3bf7627d06f5, user_id=8888c1586b224814b92ea780b9149994, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.566408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.586613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1050/2118] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:56 2022] PUT /identity/v3/projects/3650aec309f5416599fe3bf7627d06f5/users/8888c1586b224814b92ea780b9149994/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:56.587634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.627642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1069/2119] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:56.629291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.629787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.630134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.665094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] MFA Rules not processed for user `8888c1586b224814b92ea780b9149994`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:56.679235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:56.715745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:56.728761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae32a74-e850-47b0-927e-70feb3d7fd5a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.735378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.736364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.736731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.737070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.737775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.740011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.747439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1070/2120] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:56.747439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.765074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a5a6a-d068-4fc6-aa61-044995df39a4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.771497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1051/2121] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:56.772420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.775228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.775856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.776138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.776403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.777040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.778965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.827509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1071/2122] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:56.831569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.866942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475679a6-27ee-4cff-8fec-f80e05ec8800 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.906488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750775aa-dc70-4561-bb90-04b3e42e4711 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.913502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.914500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.914853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.915260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.916061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.918408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.964867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.965757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.966244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.967123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.968431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.971150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:56.977085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1072/2123] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/users => generated 281 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:56.977085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:56.988330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421911e-9848-44e2-8968-83e642d4ba5a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:56.993728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:56.994494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:56.994870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:56.995087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:56.995968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:56.997071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:56.999675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.012480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1052/2124] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/projects => generated 369 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:57.013343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.044607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1073/2125] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:56 2022] POST /identity/v3/users => generated 289 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:57.048612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.061951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385d0c7-2bf8-468c-9b36-d0f105f864b5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.112922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066289a-6389-429c-afae-67e5901011f3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.119171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.119836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.120118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.120956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.121631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.124159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.136454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1053/2126] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:57.137838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.139118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.140073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.140860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.141142 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.141779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.143734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.151117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e1654-ead7-412b-85f0-bebb99000164 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.156694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.157385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.157748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.158027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.158947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.160865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.168286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1074/2127] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:57 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:57.169228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.177624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1054/2128] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:57.180157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c6133-425d-4c67-a044-10586fcb71cc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.181555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.186342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.187027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.187516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.187799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.188459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.190225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.195435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75501da4-83bd-4c2e-9a90-bd913a23ec61 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.204026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.204244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1075/2129] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:57.205177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.207977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.208264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.208615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.209188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.210843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.217313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12f076-4dbd-43bb-a608-1faba2971b18 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.222110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1055/2130] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:57.222979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.224661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.225417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.225887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.226163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] PATH_INFO: `/v3/projects/dc1158a7542f434abfc7123628ac3b23/users/ff772e35c58d4dce82d9123f82ee4610/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.234769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5945b28-7656-4d38-950d-cdb09a56c117 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.239743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.240217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:57.240401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.240680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.241605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] PATH_INFO: `/v3/projects/f32db02762af48bb80268838e373a056/users/179e9308ae9343278b4c0d2fd4f8aa02/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.243535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1158a7542f434abfc7123628ac3b23, user_id=ff772e35c58d4dce82d9123f82ee4610, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.247100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.260931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32db02762af48bb80268838e373a056, user_id=179e9308ae9343278b4c0d2fd4f8aa02, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.264080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.269809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1076/2131] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:57 2022] PUT /identity/v3/projects/dc1158a7542f434abfc7123628ac3b23/users/ff772e35c58d4dce82d9123f82ee4610/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:57.269809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.282956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1056/2132] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:57 2022] PUT /identity/v3/projects/f32db02762af48bb80268838e373a056/users/179e9308ae9343278b4c0d2fd4f8aa02/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:57.283723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.359119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9caa2-8fd5-40c5-be4f-2b5894e842f1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.364582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906b313-38f4-4ab0-a005-65f0e0299b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.365099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906b313-38f4-4ab0-a005-65f0e0299b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.365373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906b313-38f4-4ab0-a005-65f0e0299b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.365642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906b313-38f4-4ab0-a005-65f0e0299b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.366300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906b313-38f4-4ab0-a005-65f0e0299b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.368145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906b313-38f4-4ab0-a005-65f0e0299b34 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.368537 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cff100-e59c-4ee1-b462-40e77f8245cd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.375350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.376036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.376355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.377037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] PATH_INFO: `/v3/projects/07b37c536dda48d6947cd18fdd37210a/users/6355f60e78af4be580b295cb3b49c962/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.384442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1057/2133] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:57.386073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.386364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.386638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.395962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b37c536dda48d6947cd18fdd37210a, user_id=6355f60e78af4be580b295cb3b49c962, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.400542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.419380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1077/2134] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:57 2022] PUT /identity/v3/projects/07b37c536dda48d6947cd18fdd37210a/users/6355f60e78af4be580b295cb3b49c962/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:57.420231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.422617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] MFA Rules not processed for user `179e9308ae9343278b4c0d2fd4f8aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:57.424040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:57.466937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:57.522632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aacead-bf01-43a2-96da-bdca6312e4f4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.528673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271258-7249-4e5a-8333-73391458b3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.529910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271258-7249-4e5a-8333-73391458b3e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.529910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271258-7249-4e5a-8333-73391458b3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.530121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271258-7249-4e5a-8333-73391458b3e5 admin admin] PATH_INFO: `/v3/projects/dc1158a7542f434abfc7123628ac3b23/users/ff772e35c58d4dce82d9123f82ee4610/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.532632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271258-7249-4e5a-8333-73391458b3e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1158a7542f434abfc7123628ac3b23, user_id=ff772e35c58d4dce82d9123f82ee4610, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.535447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271258-7249-4e5a-8333-73391458b3e5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.541086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1058/2135] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:57.545875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.572450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1078/2136] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:57 2022] PUT /identity/v3/projects/dc1158a7542f434abfc7123628ac3b23/users/ff772e35c58d4dce82d9123f82ee4610/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:57.573383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6d425-bc0c-4007-b46b-024cb80e3958 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.582570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.582998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.583455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.634505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] MFA Rules not processed for user `ff772e35c58d4dce82d9123f82ee4610`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:57.636408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:57.663820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.664448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.664730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.665002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.665646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.667412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.678650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1059/2137] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:57.679880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.681700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:57.715285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:57.731338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1079/2138] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:57.732981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.744738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc616a9-acdd-4225-a77b-85fab5e00863 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.749270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc4328-82f6-4a92-80c9-26055638243e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.749561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.750790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.751072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.751387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] PATH_INFO: `/v3/projects/07b37c536dda48d6947cd18fdd37210a/users/6355f60e78af4be580b295cb3b49c962/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.753382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b37c536dda48d6947cd18fdd37210a, user_id=6355f60e78af4be580b295cb3b49c962, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.754586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85919f91-c460-4649-b199-a407748ca620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.755188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919f91-c460-4649-b199-a407748ca620 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.755499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919f91-c460-4649-b199-a407748ca620 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.755790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85919f91-c460-4649-b199-a407748ca620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.756081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.766873 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1080/2139] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:57 2022] PUT /identity/v3/projects/07b37c536dda48d6947cd18fdd37210a/users/6355f60e78af4be580b295cb3b49c962/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:57.767711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.769333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:57.823540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919f91-c460-4649-b199-a407748ca620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.825120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85919f91-c460-4649-b199-a407748ca620 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.831425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd9d1a-0d60-4199-ae35-47ae4d451bdb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.834113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1060/2140] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:57.835075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.835307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.835593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.835781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.835885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.836058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.836333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:57.868068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] MFA Rules not processed for user `6355f60e78af4be580b295cb3b49c962`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:57.869284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:57.879272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:57.898510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:57.899997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:57.910920 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1081/2141] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:57.911828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.956382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1061/2142] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:57.965674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:57.977506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3542-7018-4f09-91b2-97e1cb93e9a7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:57.992535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:57.994101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:57.994801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:57.995940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:58.001893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d95d0-f3c5-4553-9d97-fd1adedf169a service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:58.010713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:58.012276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:58.012441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:58.013632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:58.014155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:58.014524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] PATH_INFO: `/v3/users/179e9308ae9343278b4c0d2fd4f8aa02` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:58.016945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=179e9308ae9343278b4c0d2fd4f8aa02)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:58.018840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:58.024752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1062/2143] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:58.061962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:58.072534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:58.081474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522bea5-2310-4aee-9941-9bfdc0f04f4f service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:58.085596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:58.086134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:58.086409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:58.086692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:58.089540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '179e9308ae9343278b4c0d2fd4f8aa02'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.100005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '179e9308ae9343278b4c0d2fd4f8aa02'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.124426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '179e9308ae9343278b4c0d2fd4f8aa02'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.126388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '179e9308ae9343278b4c0d2fd4f8aa02'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.127273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-86568d24-7700-47b4-b7ba-f6f9e76082bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '179e9308ae9343278b4c0d2fd4f8aa02'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.143375 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1082/2144] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:57 2022] DELETE /identity/v3/users/179e9308ae9343278b4c0d2fd4f8aa02 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:58.151432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:58.151880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:58.152251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:58.156949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:58.158389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:58.175345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1063/2145] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:58.177284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:58.178520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:58.251260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1083/2146] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:58.259058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:58.261110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:58.330239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5349b-7ff2-430a-b62d-7bdebeaaf0d6 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:58.394296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:58.400290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c099b3-0246-4290-a2bf-b32ccffbae66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:58.401506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c099b3-0246-4290-a2bf-b32ccffbae66 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:58.402054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c099b3-0246-4290-a2bf-b32ccffbae66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:58.402704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c099b3-0246-4290-a2bf-b32ccffbae66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:58.416273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c099b3-0246-4290-a2bf-b32ccffbae66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:58.417821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c099b3-0246-4290-a2bf-b32ccffbae66 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:58.430975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1064/2147] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:58.857662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:58.859448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:58.887533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a230f42-80d9-4609-9c97-377e0e17be7f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:58.893352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:58.893998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:58.894277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:58.894556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] PATH_INFO: `/v3/projects/f32db02762af48bb80268838e373a056` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:58.895743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32db02762af48bb80268838e373a056)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:58.897671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:58.950794 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32db02762af48bb80268838e373a056', 'request_id': 'req-290f4479-efd9-4bb7-8c00-1afd0b190032'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.961983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32db02762af48bb80268838e373a056', 'request_id': 'req-290f4479-efd9-4bb7-8c00-1afd0b190032'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.968365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32db02762af48bb80268838e373a056', 'request_id': 'req-290f4479-efd9-4bb7-8c00-1afd0b190032'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.980516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] The token cache is being invalidate because project f32db02762af48bb80268838e373a056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:58.980807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.981476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-290f4479-efd9-4bb7-8c00-1afd0b190032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:58.983170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1084/2148] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:58 2022] DELETE /identity/v3/projects/f32db02762af48bb80268838e373a056 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:58.987967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:58.988441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:58.988527 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.009528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:59.011060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:59.074331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1065/2149] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:59.082046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.082207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.082487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.111764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:59.113049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:59.123240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1085/2150] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:59.129405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.143716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971a6f6-446f-4b1b-9f86-9acf241453ae None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.148780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.149419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.149699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.149978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.150635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:59.150926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.152592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.165539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1066/2151] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:59.175555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.184365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab7c9b-1fc8-41fb-ac63-4fcffb3b1700 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.192081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.192845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.193285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.193563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.194198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.196196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.231264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1086/2152] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:59 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:59.234841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.245307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125b910-6348-42d7-bd4d-1e12dbc42d35 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.313237 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.316674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.316674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.316674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.316674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.319475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.346054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1067/2153] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:59 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:59.349930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.357041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:59.362069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41053b-6dff-4831-b588-fdd2b8d3221b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.369243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.369767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.370202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.370791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.371490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.373428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.384742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1087/2154] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:59.389109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.402096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910cc5c-58e4-4562-aafa-f87b385498f1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.407331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933ca76-ccea-4593-8bd5-f124bb164044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.407951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933ca76-ccea-4593-8bd5-f124bb164044 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.408227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933ca76-ccea-4593-8bd5-f124bb164044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.408500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933ca76-ccea-4593-8bd5-f124bb164044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.409171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933ca76-ccea-4593-8bd5-f124bb164044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.410774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933ca76-ccea-4593-8bd5-f124bb164044 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.422629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1068/2155] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:59.427021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.438235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47763-a4dd-43d0-9a31-b16566cfffaa admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.443148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.443944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.444286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.444594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] PATH_INFO: `/v3/projects/b9e966d6ba9b41909154ba63326fb5be/users/12ae1627c9a64e6a8559b09a876dffa0/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.465096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e966d6ba9b41909154ba63326fb5be, user_id=12ae1627c9a64e6a8559b09a876dffa0, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.467808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.482782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1088/2156] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:59 2022] PUT /identity/v3/projects/b9e966d6ba9b41909154ba63326fb5be/users/12ae1627c9a64e6a8559b09a876dffa0/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:59.493798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.494082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.494356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.533770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] MFA Rules not processed for user `12ae1627c9a64e6a8559b09a876dffa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:59.535317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:59.610912 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:59.639245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1069/2157] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:59.648457 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.650289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:59.699830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:59.701453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:59.709143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25f32f-44c1-41cf-8ad8-9523e54e4ff7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.712740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32760be0-420e-4017-8879-d7ec22072e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.713338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32760be0-420e-4017-8879-d7ec22072e2f service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.713613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32760be0-420e-4017-8879-d7ec22072e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.713884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32760be0-420e-4017-8879-d7ec22072e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.723676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32760be0-420e-4017-8879-d7ec22072e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.725165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32760be0-420e-4017-8879-d7ec22072e2f service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.735709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1089/2158] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:59.769559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cafa6-1c4b-4dd9-ab99-e1830543bd45 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:59.774328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:59.774953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.775261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.775543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] PATH_INFO: `/v3/users/8888c1586b224814b92ea780b9149994` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.776709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8888c1586b224814b92ea780b9149994)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:59.778670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:59.852040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8888c1586b224814b92ea780b9149994'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:59.868962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8888c1586b224814b92ea780b9149994'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:59.869641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8888c1586b224814b92ea780b9149994'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:59.882569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8888c1586b224814b92ea780b9149994'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:59.890624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8888c1586b224814b92ea780b9149994'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:59.898443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1070/2159] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:59 2022] DELETE /identity/v3/users/8888c1586b224814b92ea780b9149994 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:59.904061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:59.904526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:59.904614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:59.931628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:59.932549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:00.014309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1090/2160] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:00.025434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:00.028329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:00.144901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd77721-e322-4c84-81a0-ea7356266e76 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:00.149214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:00.149969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:00.150258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:00.155138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:00.174994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:00.176508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:00.191587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1071/2161] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:00.474498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:00.474498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:00.484759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13bb4a-a1ba-4a4e-8b81-24f133eec2a1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:00.492118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:00.492768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:00.493044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:00.493325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] PATH_INFO: `/v3/projects/3650aec309f5416599fe3bf7627d06f5` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:00.494452 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3650aec309f5416599fe3bf7627d06f5)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:00.496526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:00.545894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3650aec309f5416599fe3bf7627d06f5', 'request_id': 'req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.553409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3650aec309f5416599fe3bf7627d06f5', 'request_id': 'req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.559037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3650aec309f5416599fe3bf7627d06f5', 'request_id': 'req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.560901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:00.572778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] The token cache is being invalidate because project 3650aec309f5416599fe3bf7627d06f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:00.573069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.573772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.575289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1091/2162] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:00 2022] DELETE /identity/v3/projects/3650aec309f5416599fe3bf7627d06f5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:00.579798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:00.589470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d56738-b6c0-4f7e-8bc7-3cf217de35ee service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:00.594042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:00.652686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:00.653324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:00.653632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:00.653905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] PATH_INFO: `/v3/users/6355f60e78af4be580b295cb3b49c962` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:00.655046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6355f60e78af4be580b295cb3b49c962)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:00.656990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:00.721612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6355f60e78af4be580b295cb3b49c962'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.729533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6355f60e78af4be580b295cb3b49c962'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.730501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6355f60e78af4be580b295cb3b49c962'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.744276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6355f60e78af4be580b295cb3b49c962'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.751632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ead819c2-8459-4301-b3fa-b77f9f1109b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6355f60e78af4be580b295cb3b49c962'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:00.758462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1072/2163] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:00 2022] DELETE /identity/v3/users/6355f60e78af4be580b295cb3b49c962 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:01.060215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.061693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:01.080743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402273-8bc1-4e85-99f2-88e3b8cf43e2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.139291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.139923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.140235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.140516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] PATH_INFO: `/v3/projects/07b37c536dda48d6947cd18fdd37210a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.141616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b37c536dda48d6947cd18fdd37210a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.143574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.201550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b37c536dda48d6947cd18fdd37210a', 'request_id': 'req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:01.212252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b37c536dda48d6947cd18fdd37210a', 'request_id': 'req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:01.219270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b37c536dda48d6947cd18fdd37210a', 'request_id': 'req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:01.233467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] The token cache is being invalidate because project 07b37c536dda48d6947cd18fdd37210a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:01.233798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:01.234988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b3e3ba27-b484-4ad6-ab2a-3c9f53f5667f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:01.237395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1092/2164] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:01 2022] DELETE /identity/v3/projects/07b37c536dda48d6947cd18fdd37210a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:01.241482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.241974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.242055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.264840 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:01.266115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:01.348775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1073/2165] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:01.357623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.357909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.358206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.379030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:01.380657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:01.388934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1093/2166] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:01.393271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.403055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147d9ff-1911-41e1-8b92-e1f09624cbec None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.408303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.408928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.409262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.409478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.410132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:01.410438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.412424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.424657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1074/2167] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:01.429049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.438646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374bb66-0853-4b61-afe7-3b8797c7a7fb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.444060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.444829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.445040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.445199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.445830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.447905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.483328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1094/2168] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:01.485418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.497841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8261cd-475d-4f85-b83a-4a29de50e4d2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.512549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:01.560478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.561093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.561374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.561650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.562340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.564309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.587037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1075/2169] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:01.591721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.602944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816aeaf1-0e26-4f63-8d29-d91ba78b4073 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.611224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.612463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.612805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.613086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.614045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.616154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.627587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1095/2170] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:01.631387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.642080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f9e0f-5f10-4ab4-9e08-94ceb13ba1e4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.646602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22c0d3-6779-4402-b461-8e9d38322b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.647210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22c0d3-6779-4402-b461-8e9d38322b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.647506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22c0d3-6779-4402-b461-8e9d38322b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.647779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22c0d3-6779-4402-b461-8e9d38322b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.648412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22c0d3-6779-4402-b461-8e9d38322b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.650007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22c0d3-6779-4402-b461-8e9d38322b1c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.662162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1076/2171] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:01.666116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.676214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d065f2-a11f-4987-811c-dd9754abecbe admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.682799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.683488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.683769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.684042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] PATH_INFO: `/v3/projects/92b347d6b3c346f593ca4ba9baf627d0/users/e053a3001f9144e38e13bf1a86154d3b/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.698335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b347d6b3c346f593ca4ba9baf627d0, user_id=e053a3001f9144e38e13bf1a86154d3b, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.701060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.714001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1096/2172] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:01 2022] PUT /identity/v3/projects/92b347d6b3c346f593ca4ba9baf627d0/users/e053a3001f9144e38e13bf1a86154d3b/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:01.725262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.725821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.726344 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.756959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] MFA Rules not processed for user `e053a3001f9144e38e13bf1a86154d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:01.758170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:01.826111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1077/2173] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:01.831336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.856574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:01.896077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c4bb3-44c1-4f78-b6ee-a24186f73fac admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:01.900800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:01.901407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:01.901687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:01.901965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:01.902602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:01.904551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:01.934405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1097/2174] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:01.939272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:01.955139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff856bd-c23a-4371-bdf7-944ed23ba618 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.026355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.026967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.027271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.027735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.028400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.030311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.062577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1078/2175] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:01 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:02.063740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.075121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191e67-cdce-4a98-9ab9-f313f2986f1b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.080425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.081021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.081297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.081570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.082652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.083836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.095029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1098/2176] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:02.099134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.109860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fe356-313b-4af1-9728-e6dc60097c5c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.115905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.116809 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.117125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.117512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] PATH_INFO: `/v3/projects/7544e59fd2b04b62b370f68083a3abc0/users/91cf5e2198054c1cab921b818eef4acf/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.130208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:02.136854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7544e59fd2b04b62b370f68083a3abc0, user_id=91cf5e2198054c1cab921b818eef4acf, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.139780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.156706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1079/2177] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:02 2022] PUT /identity/v3/projects/7544e59fd2b04b62b370f68083a3abc0/users/91cf5e2198054c1cab921b818eef4acf/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:02.160793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.170916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5bf82-c2f6-46f4-8893-cde83e3b928e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.246354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9233a-1225-4370-9511-bfbbbf14c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.247331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9233a-1225-4370-9511-bfbbbf14c4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.247994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9233a-1225-4370-9511-bfbbbf14c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.248468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9233a-1225-4370-9511-bfbbbf14c4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.249291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9233a-1225-4370-9511-bfbbbf14c4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.252078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9233a-1225-4370-9511-bfbbbf14c4ad admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.271255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1099/2178] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:02.273185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.286828 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed70902-7bf4-4f27-8c0e-30b99a3acc2f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.291886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.292561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.292847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.293119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] PATH_INFO: `/v3/projects/7544e59fd2b04b62b370f68083a3abc0/users/91cf5e2198054c1cab921b818eef4acf/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.295096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7544e59fd2b04b62b370f68083a3abc0, user_id=91cf5e2198054c1cab921b818eef4acf, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.298974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.320587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1080/2179] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:02 2022] PUT /identity/v3/projects/7544e59fd2b04b62b370f68083a3abc0/users/91cf5e2198054c1cab921b818eef4acf/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:02.327075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.327528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.327934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.382079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] MFA Rules not processed for user `91cf5e2198054c1cab921b818eef4acf`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:02.383349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:02.522453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1100/2180] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:02.530330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.532678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:02.616969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0159b-e7a6-49eb-a4c8-ea958346fdbc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.620881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c5080-bf55-4edd-abea-f12ee2ffc2e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.621490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c5080-bf55-4edd-abea-f12ee2ffc2e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.621769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c5080-bf55-4edd-abea-f12ee2ffc2e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.622045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c5080-bf55-4edd-abea-f12ee2ffc2e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.631884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c5080-bf55-4edd-abea-f12ee2ffc2e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.633404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c5080-bf55-4edd-abea-f12ee2ffc2e4 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.644274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1081/2181] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:02.722064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.760444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:02.818108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff72e6a-87da-4223-96ec-820e1ca46b2d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.823280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.823923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.824251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.824527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] PATH_INFO: `/v3/users/e053a3001f9144e38e13bf1a86154d3b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.825729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e053a3001f9144e38e13bf1a86154d3b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.828234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:02.905665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e053a3001f9144e38e13bf1a86154d3b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:02.917440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e053a3001f9144e38e13bf1a86154d3b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:02.934706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e053a3001f9144e38e13bf1a86154d3b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:02.935694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e053a3001f9144e38e13bf1a86154d3b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:02.936407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e053a3001f9144e38e13bf1a86154d3b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:02.952305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1101/2182] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:02 2022] DELETE /identity/v3/users/e053a3001f9144e38e13bf1a86154d3b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:02.956985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.957145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.957444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.981913 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:02.983110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.084539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1082/2183] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:03.096580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:03.098679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.176566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed483bd-7004-4a94-b2bd-a1de493ba89f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:03.180827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:03.181453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.181787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.182053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.191724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:03.193897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:03.204701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1102/2184] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:03.485032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:03.486641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.501825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9114b-6e74-4ef4-8792-dc17432c42ad None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:03.507865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:03.508576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.508758 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.509074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] PATH_INFO: `/v3/projects/92b347d6b3c346f593ca4ba9baf627d0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.510402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b347d6b3c346f593ca4ba9baf627d0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:03.512605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:03.560185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b347d6b3c346f593ca4ba9baf627d0', 'request_id': 'req-31377182-4881-4e37-a580-0ea83391855b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.568081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b347d6b3c346f593ca4ba9baf627d0', 'request_id': 'req-31377182-4881-4e37-a580-0ea83391855b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.573952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b347d6b3c346f593ca4ba9baf627d0', 'request_id': 'req-31377182-4881-4e37-a580-0ea83391855b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.586018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] The token cache is being invalidate because project 92b347d6b3c346f593ca4ba9baf627d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:03.586297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.586958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.592247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1083/2185] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:03 2022] DELETE /identity/v3/projects/92b347d6b3c346f593ca4ba9baf627d0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:03.593071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:03.602947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d7d8c-bfe6-4284-b192-f753c8c45d85 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:03.645237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:03.669331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:03.670372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.671000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.671000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] PATH_INFO: `/v3/users/91cf5e2198054c1cab921b818eef4acf` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.672218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] RBAC: Authorizing `identity:delete_user(user_id=91cf5e2198054c1cab921b818eef4acf)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:03.674261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:03.725671 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91cf5e2198054c1cab921b818eef4acf'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.735653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91cf5e2198054c1cab921b818eef4acf'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.750113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91cf5e2198054c1cab921b818eef4acf'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.751012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91cf5e2198054c1cab921b818eef4acf'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.751791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91cf5e2198054c1cab921b818eef4acf'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.764528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1103/2186] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:03 2022] DELETE /identity/v3/users/91cf5e2198054c1cab921b818eef4acf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:04.133262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.134858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.143139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:04.148803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31377182-4881-4e37-a580-0ea83391855b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.191077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.192541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.227979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.228631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.228948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.229171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] PATH_INFO: `/v3/projects/7544e59fd2b04b62b370f68083a3abc0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.230296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7544e59fd2b04b62b370f68083a3abc0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.232246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.262242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5754-5fc8-4b32-8ee9-5746f0b951af admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.267809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.268470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.268750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.269020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] PATH_INFO: `/v3/users/12ae1627c9a64e6a8559b09a876dffa0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.270264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ae1627c9a64e6a8559b09a876dffa0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.272334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.292103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7544e59fd2b04b62b370f68083a3abc0', 'request_id': 'req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.302857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7544e59fd2b04b62b370f68083a3abc0', 'request_id': 'req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.315482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7544e59fd2b04b62b370f68083a3abc0', 'request_id': 'req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.328838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] The token cache is being invalidate because project 7544e59fd2b04b62b370f68083a3abc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:04.329133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.329777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d4d22bc2-decb-4c7c-9ecc-104bfeb81269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.331361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1084/2187] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:04 2022] DELETE /identity/v3/projects/7544e59fd2b04b62b370f68083a3abc0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:04.336153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cafde0-e7f9-4848-8c13-74939cdc4823 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.336501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cafde0-e7f9-4848-8c13-74939cdc4823 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.336869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cafde0-e7f9-4848-8c13-74939cdc4823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.338153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ae1627c9a64e6a8559b09a876dffa0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.347634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ae1627c9a64e6a8559b09a876dffa0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.355049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:04.360029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-98cafde0-e7f9-4848-8c13-74939cdc4823 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:04.361395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-98cafde0-e7f9-4848-8c13-74939cdc4823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.364982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ae1627c9a64e6a8559b09a876dffa0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.365998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ae1627c9a64e6a8559b09a876dffa0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.367245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6ede8e2e-5bca-44fe-9cd2-7dcd8508ce57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ae1627c9a64e6a8559b09a876dffa0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.388888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1104/2188] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:04 2022] DELETE /identity/v3/users/12ae1627c9a64e6a8559b09a876dffa0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:04.393239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.393526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.393804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.414530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:04.415799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.456690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1085/2189] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:04.470236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.470535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.470818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.486517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1105/2190] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:04.487422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.491439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.497591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:04.498827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.507184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1086/2191] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:04.508103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.508652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bf99e-65aa-4c74-9a79-ec94eb1d9a01 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.512144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.514243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.514882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.515760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.516062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] PATH_INFO: `/v3/users/39ce0b6582a544d893c857e36e2eecf2` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.517202 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ce0b6582a544d893c857e36e2eecf2)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.519251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.544738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:04.569985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ce0b6582a544d893c857e36e2eecf2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.575650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114abb4-63db-48b9-b035-38b0667c9e79 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.578921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ce0b6582a544d893c857e36e2eecf2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.580593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.581208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.581484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.581753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.590999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.592458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.592689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ce0b6582a544d893c857e36e2eecf2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.593922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ce0b6582a544d893c857e36e2eecf2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.594561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c8fb5c56-ee08-49a4-adcb-d4608e8706ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ce0b6582a544d893c857e36e2eecf2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.603320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1087/2192] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:04.603320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.608322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1106/2193] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:04 2022] DELETE /identity/v3/users/39ce0b6582a544d893c857e36e2eecf2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:04.616523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b59f4f-7d26-4f82-a6ce-34ba220a677a service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.619538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.620025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.620308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.641664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:04.642911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.682735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.683391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.683776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.685005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.685005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:04.685274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.686905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.701080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:04.709919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1088/2194] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:04.715775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.721574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.731293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1107/2195] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:04.736533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9c10-4fca-40b8-915a-10efa126c170 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.743100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.744068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.744530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.745159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.746032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.747493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.748692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.752396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.795388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1089/2196] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:04.798850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.876469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74617bcb-f8bc-4b52-bd80-4b87610900b9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.910861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1db88-2560-4f05-9cc7-218183bfec25 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.916403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.917042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.917223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.917719 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.918893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.921210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.947986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1090/2197] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:04.952051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.975651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0507-b71b-4679-a590-e5e325da7b7e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.977761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.979322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.980012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.980978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.981477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.982186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.982642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.982841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.983682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.985974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.002424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1091/2198] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:05.006067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.006213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.009691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.010757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.025769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1108/2199] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:00:05.026957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.030957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0a791-ae05-4785-b829-470bbdb7ca00 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.036574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.037328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.037698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.038044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] PATH_INFO: `/v3/projects/b9e966d6ba9b41909154ba63326fb5be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.038634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640793d-7801-4462-b30d-27f26d4662d0 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.039563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e966d6ba9b41909154ba63326fb5be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.041936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.043685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.044299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.044582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.044855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.046212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.047856 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.062952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1109/2200] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:05.065166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.075811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:05.079335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a40eb-dc96-44ac-bb17-102521ddf22d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.085068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d502b67-1b89-4e76-aef5-45c405d89588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.085854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d502b67-1b89-4e76-aef5-45c405d89588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.086340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d502b67-1b89-4e76-aef5-45c405d89588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.086566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d502b67-1b89-4e76-aef5-45c405d89588 admin admin] PATH_INFO: `/v3/projects/90df659dd1db4ceda249541b60ee5b64/users/b60dcabb539f4b41b532d61a38b2a5c0/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.103965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d502b67-1b89-4e76-aef5-45c405d89588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90df659dd1db4ceda249541b60ee5b64, user_id=b60dcabb539f4b41b532d61a38b2a5c0, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.106686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d502b67-1b89-4e76-aef5-45c405d89588 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.116489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e966d6ba9b41909154ba63326fb5be', 'request_id': 'req-875321d9-6643-4302-948e-1e76f693775c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.122739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1110/2201] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:05 2022] PUT /identity/v3/projects/90df659dd1db4ceda249541b60ee5b64/users/b60dcabb539f4b41b532d61a38b2a5c0/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:05.134212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.134503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.134817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.137635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e966d6ba9b41909154ba63326fb5be', 'request_id': 'req-875321d9-6643-4302-948e-1e76f693775c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.160130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e966d6ba9b41909154ba63326fb5be', 'request_id': 'req-875321d9-6643-4302-948e-1e76f693775c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.175662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] The token cache is being invalidate because project b9e966d6ba9b41909154ba63326fb5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:05.175977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.180398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-875321d9-6643-4302-948e-1e76f693775c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.182498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1092/2202] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:05 2022] DELETE /identity/v3/projects/b9e966d6ba9b41909154ba63326fb5be => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:05.188011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ae493-eb70-47c9-9e5b-13e6e907581b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.188526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ae493-eb70-47c9-9e5b-13e6e907581b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.188854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ae493-eb70-47c9-9e5b-13e6e907581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.201816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] MFA Rules not processed for user `b60dcabb539f4b41b532d61a38b2a5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:05.205039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.237314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8b7ae493-eb70-47c9-9e5b-13e6e907581b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:05.239136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8b7ae493-eb70-47c9-9e5b-13e6e907581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.302944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1111/2203] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:05.311827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.314015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.320545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:05.323266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1093/2204] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:05.331351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.331641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.331961 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.357363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:05.358622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.368578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1094/2205] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:05.372563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.383243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a647cb-4ef8-4db2-ac6f-ebf1e48b6736 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.389414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b61cd-a0ab-4b5e-90e6-8c39d69fb763 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.393631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.394543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.394825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.395098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.396240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.396959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.397275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.397555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.398456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:05.398687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.400353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.404848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.406316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.415491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1095/2206] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:00:05.416290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1112/2207] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:00:05.418483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.430587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9c9b6-4a2b-4950-ac9f-e11ca2fd98ff admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.438524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.440467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.441542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.442396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.442717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.443016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.443620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.446225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.456073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da1468-f2aa-411d-a218-e929e0cc410e service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.462185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.462824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.463106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.463416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] PATH_INFO: `/v3/users/b60dcabb539f4b41b532d61a38b2a5c0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.464570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60dcabb539f4b41b532d61a38b2a5c0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.466986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.496262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1096/2208] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:05.497866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.549947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60dcabb539f4b41b532d61a38b2a5c0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.564329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60dcabb539f4b41b532d61a38b2a5c0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.597227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60dcabb539f4b41b532d61a38b2a5c0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.598208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60dcabb539f4b41b532d61a38b2a5c0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.599245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b684616f-97eb-4bb2-bf2a-260d561fcf94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60dcabb539f4b41b532d61a38b2a5c0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.612588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b4c05-2c91-472e-9002-b6fabc6a67d0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.615523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1113/2209] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:05 2022] DELETE /identity/v3/users/b60dcabb539f4b41b532d61a38b2a5c0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:05.622584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.622584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.623107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.626700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:05.651221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:05.652945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.727544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.728373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.728636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.729774 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.730443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.735320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.770884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1097/2210] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:05.772052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.773177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1114/2211] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:05.776638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.778053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.780126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.794928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654badc-3385-4759-aeb5-a320afae1ade admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.804341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.805673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.805965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.806239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] PATH_INFO: `/v3/projects/6e75447b10894f0c92260947a00a060d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.807471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e75447b10894f0c92260947a00a060d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.810182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.812565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f0dfd-ac51-4e4d-a74f-e67288f59646 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.818306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.819013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.819297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.819586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.820352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.822270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.848592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1115/2212] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:05.849558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.851618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.865740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:05.904256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e75447b10894f0c92260947a00a060d', 'request_id': 'req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.916435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e75447b10894f0c92260947a00a060d', 'request_id': 'req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.924313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e75447b10894f0c92260947a00a060d', 'request_id': 'req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.933919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51539-56f8-4d88-b313-4f995895b785 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.951953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] The token cache is being invalidate because project 6e75447b10894f0c92260947a00a060d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:05.952253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.953112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:05.954706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1098/2213] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:05 2022] DELETE /identity/v3/projects/6e75447b10894f0c92260947a00a060d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:05.955665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.956980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.994134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.994667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.994947 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.995363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.997833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.025718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ea2f-a26e-438e-aeb4-58a21a9ce56c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.030711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.032163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.032450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.032773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.033417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.035053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.045927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1099/2214] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:06.047891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.049868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.062491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573ce56-5701-461b-b729-9a31294df60f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.064646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.066167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.067104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.067742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.068021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.068291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] PATH_INFO: `/v3/users/ff772e35c58d4dce82d9123f82ee4610` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.069528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff772e35c58d4dce82d9123f82ee4610)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.071483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.077143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1116/2215] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:06.078512 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.085641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:06.088627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c734014-c337-480b-83e2-b78bfb04323d service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.094828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f3757-28d6-41e9-a069-89e456756217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.095534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3757-28d6-41e9-a069-89e456756217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.095815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3757-28d6-41e9-a069-89e456756217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.096090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3757-28d6-41e9-a069-89e456756217 admin admin] PATH_INFO: `/v3/projects/60280bb8b9a647d18d184a06b0b60612/users/8dfef6201aef4087b0dfb9751fe4ea0e/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.116801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f3757-28d6-41e9-a069-89e456756217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60280bb8b9a647d18d184a06b0b60612, user_id=8dfef6201aef4087b0dfb9751fe4ea0e, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.120127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f3757-28d6-41e9-a069-89e456756217 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.134139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff772e35c58d4dce82d9123f82ee4610'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.135355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1117/2216] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:06 2022] PUT /identity/v3/projects/60280bb8b9a647d18d184a06b0b60612/users/8dfef6201aef4087b0dfb9751fe4ea0e/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:06.144432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff772e35c58d4dce82d9123f82ee4610'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.145700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff772e35c58d4dce82d9123f82ee4610'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.147904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.148201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.148522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.168533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff772e35c58d4dce82d9123f82ee4610'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.181328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff772e35c58d4dce82d9123f82ee4610'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.181823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] MFA Rules not processed for user `8dfef6201aef4087b0dfb9751fe4ea0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:06.183300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.190742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1100/2217] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:06 2022] DELETE /identity/v3/users/ff772e35c58d4dce82d9123f82ee4610 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:06.251409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:06.311316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1118/2218] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:06.315730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.317332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.428955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3526345-e35e-420b-8df8-0124c969c3c1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.439530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.440144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.440421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.440691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.441369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.443386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.474329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1101/2219] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:06 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:06.479483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.576860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.579434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.604519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76b8e-64cb-4d31-b240-e89dbc7deb38 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.697218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.697959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.698310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.698656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.699596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.701957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.707895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48572b-b0e4-479f-90fb-2df5addacdef admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.713490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.714862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.714862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.715348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] PATH_INFO: `/v3/projects/90df659dd1db4ceda249541b60ee5b64` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.716750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90df659dd1db4ceda249541b60ee5b64)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.719582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.735621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1119/2220] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:06 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:06.741052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.749965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:06.754453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f608d-1f39-4bf0-9782-3556bed86f87 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.762072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.762072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.762072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.762072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.762706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.764760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.777500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1120/2221] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:06.782970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.784984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.800548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb88f8-d31c-48a8-811c-4492fcf08446 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.816722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90df659dd1db4ceda249541b60ee5b64', 'request_id': 'req-4fe32377-e9e0-4022-bb22-7f7d49564463'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.824598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90df659dd1db4ceda249541b60ee5b64', 'request_id': 'req-4fe32377-e9e0-4022-bb22-7f7d49564463'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.831706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90df659dd1db4ceda249541b60ee5b64', 'request_id': 'req-4fe32377-e9e0-4022-bb22-7f7d49564463'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.842882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] The token cache is being invalidate because project 90df659dd1db4ceda249541b60ee5b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:06.844798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.845532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.847054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1102/2222] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:06 2022] DELETE /identity/v3/projects/90df659dd1db4ceda249541b60ee5b64 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:06.851310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:06.853107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.861658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.862289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.862568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.862840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] PATH_INFO: `/v3/projects/dc1158a7542f434abfc7123628ac3b23` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.864174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1158a7542f434abfc7123628ac3b23)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.866157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:06.874667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:06.920549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1158a7542f434abfc7123628ac3b23', 'request_id': 'req-195333df-4936-4783-8a40-0170ea60ab35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.925870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe32377-e9e0-4022-bb22-7f7d49564463 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:06.928440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1158a7542f434abfc7123628ac3b23', 'request_id': 'req-195333df-4936-4783-8a40-0170ea60ab35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.934469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1158a7542f434abfc7123628ac3b23', 'request_id': 'req-195333df-4936-4783-8a40-0170ea60ab35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.947160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] The token cache is being invalidate because project dc1158a7542f434abfc7123628ac3b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:06.947479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.948149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-195333df-4936-4783-8a40-0170ea60ab35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:06.949700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1121/2223] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:06 2022] DELETE /identity/v3/projects/dc1158a7542f434abfc7123628ac3b23 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:06.950855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.951136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.951456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.971851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:06.972946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:06.983523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d30de-98fa-42cc-ad88-97db5a6eca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:06.984167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d30de-98fa-42cc-ad88-97db5a6eca59 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:06.984443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d30de-98fa-42cc-ad88-97db5a6eca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:06.984712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d30de-98fa-42cc-ad88-97db5a6eca59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:06.985350 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d30de-98fa-42cc-ad88-97db5a6eca59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:06.987002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d30de-98fa-42cc-ad88-97db5a6eca59 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.003892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1103/2224] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:07.003892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cb331-2e84-41bd-b167-838c01caac50 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.003892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cb331-2e84-41bd-b167-838c01caac50 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.003892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cb331-2e84-41bd-b167-838c01caac50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.024926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7a2cb331-2e84-41bd-b167-838c01caac50 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:07.026146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7a2cb331-2e84-41bd-b167-838c01caac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.037486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1122/2226] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:07.037678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1104/2226] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:07.038041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.039368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.045615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf25d2-575b-4cb1-8c69-5f26f68a1013 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.045914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf25d2-575b-4cb1-8c69-5f26f68a1013 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.046681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf25d2-575b-4cb1-8c69-5f26f68a1013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.052420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f082f-fa00-4837-85d2-667e5db4f454 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.058665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.060012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.060336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.060542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] PATH_INFO: `/v3/projects/bd79e937afca45aca6dba157ea05c944/users/9d31bb74ad2945dc8ba631d8be316179/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.071173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3caf25d2-575b-4cb1-8c69-5f26f68a1013 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:07.072353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3caf25d2-575b-4cb1-8c69-5f26f68a1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.076946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd79e937afca45aca6dba157ea05c944, user_id=9d31bb74ad2945dc8ba631d8be316179, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.080225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.085453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1105/2227] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:07.087101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0211e1a-e3f0-4d2f-b4be-8c65ab427bdf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.087483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0211e1a-e3f0-4d2f-b4be-8c65ab427bdf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.087785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0211e1a-e3f0-4d2f-b4be-8c65ab427bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.096173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1123/2228] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:07 2022] PUT /identity/v3/projects/bd79e937afca45aca6dba157ea05c944/users/9d31bb74ad2945dc8ba631d8be316179/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:07.096999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.119646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f0211e1a-e3f0-4d2f-b4be-8c65ab427bdf None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:07.120432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f0211e1a-e3f0-4d2f-b4be-8c65ab427bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.154618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:07.177872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7be21b-4da4-4e6a-b004-913686a2c508 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.184985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.185611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.185962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.186254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.187631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:07.187770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.190175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.194213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1106/2229] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:07.199287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.199594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.199911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.206523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1124/2230] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:07.207411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.219998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3ca21-99c4-4447-9483-805d37e7760f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.224805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.225412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.225693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.225967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.226612 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:07.226919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.228563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.230224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:07.233505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] MFA Rules not processed for user `9d31bb74ad2945dc8ba631d8be316179`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:07.234847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.249180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1125/2231] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:07.250831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.261972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5a8df-77ab-4fa7-ab00-f8ce1a6b220b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.267769 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.268379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.268660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.268933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.269565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.271548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.298884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1126/2232] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:07.299778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.307513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1107/2233] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:07.308437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.320696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c78f9-bc5a-4c80-8022-b06185fa9aae None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.369185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fcc4e-768f-4c41-a25a-33f6fd5ed6ae admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.377333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.378004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.378374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.379442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.380415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.382704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.389647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.391052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.391424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.392048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.392686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.394761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.420100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1127/2234] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:07.420941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.429037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1108/2235] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/users => generated 284 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:07.430259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.444704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c80b99-ff5e-4852-b2e7-06afa00cefe0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.502086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223348-10fb-415b-ac76-c4009177247a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.503747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:07.508062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.508693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.509149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.509337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.510230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.512195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.515074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.515734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.516018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.516351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.517000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.518941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.545509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1109/2236] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:07.545509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.547441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1128/2237] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:07.548642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.553764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146c547-d771-4197-bd17-f090d2c338a2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.601017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:07.618652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.619445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.619800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.620090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.620884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.622696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.623989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaad0b2-ff1f-4b4c-a130-78b0c7ae7e8a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.630444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.631266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.631548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.631819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.632450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.634075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.637831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1110/2238] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:07.637831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.645712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1129/2239] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:07.646715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.648293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5f4a-aaed-4653-b213-02fe130c155a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.653519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.654126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.654391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.654667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.655328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.656468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227d22-5baf-4df6-bbb5-9d5542a47f7e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.657360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.661446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.662182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.662464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.662740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.663532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.665782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.678643 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1130/2240] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:07.680917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1111/2241] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/users => generated 299 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:07.682916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.683035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.693327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e5c9e-7464-4f91-b816-8807a063d4f4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.693477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86aa71-d023-4f7f-a012-4c957caeb260 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.698036 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.698171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.698642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.698744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.698923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.699034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.699191 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] PATH_INFO: `/v3/projects/a5908d1ed3d54f38a312fe4ee0208e76/users/9959f9018f224e1983a2a3775f2b79a3/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.699325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.699848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.701454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.711747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1131/2242] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:07.712596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.716836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5908d1ed3d54f38a312fe4ee0208e76, user_id=9959f9018f224e1983a2a3775f2b79a3, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.719583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.724948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c2f2-2c23-459b-a9f4-938b8938eaa1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.730576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b79dc5-8bda-4969-b09b-cbf112fc4f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.731332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b79dc5-8bda-4969-b09b-cbf112fc4f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.731695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b79dc5-8bda-4969-b09b-cbf112fc4f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.732080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b79dc5-8bda-4969-b09b-cbf112fc4f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.732729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b79dc5-8bda-4969-b09b-cbf112fc4f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.736545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b79dc5-8bda-4969-b09b-cbf112fc4f1c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.739616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1112/2243] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:07 2022] PUT /identity/v3/projects/a5908d1ed3d54f38a312fe4ee0208e76/users/9959f9018f224e1983a2a3775f2b79a3/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:07.740586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.748255 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1132/2244] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:07.752971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.753262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.753665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.784079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] MFA Rules not processed for user `9959f9018f224e1983a2a3775f2b79a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:07.785285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.834048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c50c2-f923-48bb-9916-7b3bb80f4662 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.840138 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.840895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.841209 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.841487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] PATH_INFO: `/v3/projects/17805705e6b846d3aa34a351fff8d319/users/036e2ebd132a4db4b196da8728cd55fc/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.862224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1133/2245] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:07.863387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.865314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17805705e6b846d3aa34a351fff8d319, user_id=036e2ebd132a4db4b196da8728cd55fc, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.869898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.879793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebe22f-c4f8-43d9-91c4-1d6c60c21d44 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.886480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eea37e-1edf-483b-9ddf-d862db79575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:07.887488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eea37e-1edf-483b-9ddf-d862db79575d admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.887913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eea37e-1edf-483b-9ddf-d862db79575d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.888346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eea37e-1edf-483b-9ddf-d862db79575d admin admin] PATH_INFO: `/v3/projects/887b6c9a7444430388e541b985d53035/users/9c5a813e4833422da61dbe928e420e89/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.895935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1113/2246] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:07 2022] PUT /identity/v3/projects/17805705e6b846d3aa34a351fff8d319/users/036e2ebd132a4db4b196da8728cd55fc/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:07.896866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.898509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.911040 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eea37e-1edf-483b-9ddf-d862db79575d admin admin] RBAC: Authorizing `identity:create_grant(project_id=887b6c9a7444430388e541b985d53035, user_id=9c5a813e4833422da61dbe928e420e89, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:07.914430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eea37e-1edf-483b-9ddf-d862db79575d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:07.930908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:07.934695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1134/2247] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:07 2022] PUT /identity/v3/projects/887b6c9a7444430388e541b985d53035/users/9c5a813e4833422da61dbe928e420e89/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:07.934695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:07.934695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:07.934695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:07.969323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] MFA Rules not processed for user `036e2ebd132a4db4b196da8728cd55fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:07.970163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f092-6970-4547-b69b-6b04dfe8fc3d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:07.970696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.024561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.025214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.025549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.025829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.054109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.068007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1135/2248] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:08.069444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.107909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.109790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.120961 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1114/2249] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:08.122109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.147522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab15f39-9e7d-4f95-bd81-0ffa526c63d6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.153217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.153990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.154342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.154687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.155465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.157732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.169340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1136/2250] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:08.173954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.185244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd5e05-1510-4001-bb7b-c89e12b4780d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.191556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007216-4b59-474f-b813-4ecccfd34b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.192064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007216-4b59-474f-b813-4ecccfd34b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.192346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007216-4b59-474f-b813-4ecccfd34b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.192624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007216-4b59-474f-b813-4ecccfd34b0b admin admin] PATH_INFO: `/v3/projects/887b6c9a7444430388e541b985d53035/users/9c5a813e4833422da61dbe928e420e89/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.195008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4007216-4b59-474f-b813-4ecccfd34b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=887b6c9a7444430388e541b985d53035, user_id=9c5a813e4833422da61dbe928e420e89, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.199695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4007216-4b59-474f-b813-4ecccfd34b0b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.215256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1137/2251] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:08 2022] PUT /identity/v3/projects/887b6c9a7444430388e541b985d53035/users/9c5a813e4833422da61dbe928e420e89/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:08.226229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.226468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.226730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.246866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98db2-6695-4d62-87bb-2f1cf1a135bd service glance] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.263301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] MFA Rules not processed for user `9c5a813e4833422da61dbe928e420e89`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:08.266629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.291631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.312432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.313045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.313431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.313603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.314335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.316323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.345516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1115/2252] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:08.349680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.356508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1138/2253] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:08.368172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.370041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.435620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bacc6-042e-4b81-b26e-5bcf26c992e2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.440804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1569c2d-9ec1-4d4f-b5f1-5ba7e9de1e43 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.442470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.443088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.443399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.443670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.444485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.447123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.465270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.469362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1116/2254] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:08.474518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.488120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0360ee6-ccb1-411d-834c-85e9bab01a33 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.494494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.495095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.495407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.495680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.496325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.497939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.502420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.503296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.503731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.504173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.507621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.510553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1117/2255] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:08.514973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.527699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce591a2-bff3-4293-a778-d67823b84b84 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.533790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.534446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.534754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.535526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] PATH_INFO: `/v3/projects/59518401f11d4444abf7b3dfb5609e03/users/5723409f99c946c181dc9e8de52cd4d8/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.555168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59518401f11d4444abf7b3dfb5609e03, user_id=5723409f99c946c181dc9e8de52cd4d8, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.558119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.576283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1118/2256] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:08 2022] PUT /identity/v3/projects/59518401f11d4444abf7b3dfb5609e03/users/5723409f99c946c181dc9e8de52cd4d8/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:08.580897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.595692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.599407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.613123 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1139/2257] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:00:08.674252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86cb64-239f-4883-ab42-390cb6c7224c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.680332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a5fbc-1f4e-40f8-99fd-724e5bbcffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.681298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a5fbc-1f4e-40f8-99fd-724e5bbcffba admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.681475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a5fbc-1f4e-40f8-99fd-724e5bbcffba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.681734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a5fbc-1f4e-40f8-99fd-724e5bbcffba admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.682436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a5fbc-1f4e-40f8-99fd-724e5bbcffba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.684383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a5fbc-1f4e-40f8-99fd-724e5bbcffba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.696952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1119/2258] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:08.704060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.727885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513ad7c-76e1-4514-817f-9bedbd4d2986 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.734317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.738653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.738888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.739384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] PATH_INFO: `/v3/projects/59518401f11d4444abf7b3dfb5609e03/users/5723409f99c946c181dc9e8de52cd4d8/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.741736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59518401f11d4444abf7b3dfb5609e03, user_id=5723409f99c946c181dc9e8de52cd4d8, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.744725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.758139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1140/2259] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:08 2022] PUT /identity/v3/projects/59518401f11d4444abf7b3dfb5609e03/users/5723409f99c946c181dc9e8de52cd4d8/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:08.770121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.770438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.770715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.815901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] MFA Rules not processed for user `5723409f99c946c181dc9e8de52cd4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:08.817313 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.822656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.912517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1120/2260] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:08.924480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.926286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.998413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3f83-92a9-4be0-a9c3-3fd42c23596e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:09.004754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:09.005705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.006073 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.006433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.017573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:09.019112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.030178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1141/2261] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:09.135888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:09.145498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceaf9e-8a6c-4e24-a419-7a5081dc4104 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:09.149086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4bb09-08dc-46c6-b8a5-a5cfffa3cf89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:09.149678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4bb09-08dc-46c6-b8a5-a5cfffa3cf89 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.149955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4bb09-08dc-46c6-b8a5-a5cfffa3cf89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.150229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4bb09-08dc-46c6-b8a5-a5cfffa3cf89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.212614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4bb09-08dc-46c6-b8a5-a5cfffa3cf89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:09.214192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4bb09-08dc-46c6-b8a5-a5cfffa3cf89 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.223339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1121/2262] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:09.520274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:09.587884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d4a61-242e-4232-8b34-accec9394094 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:09.592650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:09.593801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.594081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.594345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] PATH_INFO: `/v3/users/036e2ebd132a4db4b196da8728cd55fc` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.602632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=036e2ebd132a4db4b196da8728cd55fc)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:09.604645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.616646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:09.654306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036e2ebd132a4db4b196da8728cd55fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:09.663709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036e2ebd132a4db4b196da8728cd55fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:09.677627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036e2ebd132a4db4b196da8728cd55fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:09.678366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036e2ebd132a4db4b196da8728cd55fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:09.678974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036e2ebd132a4db4b196da8728cd55fc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:09.691731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1142/2263] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:09 2022] DELETE /identity/v3/users/036e2ebd132a4db4b196da8728cd55fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:09.696501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.696994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.697105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.716540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:09.717941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:09.803042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1122/2264] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:09.813821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:09.815270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:09.886001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb788065-f93d-4ac1-a305-4e054989a29b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:09.894195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:09.894825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.895104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.895402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.905983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:09.908411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.927420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1143/2265] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:10.331970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:10.333474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:10.340798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:10.342497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:10.357602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bb09e-f45e-4f57-846b-740b32dc901e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:10.380232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:10.381107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:10.381547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:10.381983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] PATH_INFO: `/v3/projects/17805705e6b846d3aa34a351fff8d319` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:10.383687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17805705e6b846d3aa34a351fff8d319)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:10.386061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:10.388162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:10.414761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d39328-b5a3-4ab2-8f29-d0e3e167dca7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:10.418606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:10.419151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:10.419875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:10.419875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:10.422251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:10.458413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17805705e6b846d3aa34a351fff8d319', 'request_id': 'req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.470516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17805705e6b846d3aa34a351fff8d319', 'request_id': 'req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.479959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17805705e6b846d3aa34a351fff8d319', 'request_id': 'req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.496203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] The token cache is being invalidate because project 17805705e6b846d3aa34a351fff8d319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:10.496640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.497458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.499243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1144/2266] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:10 2022] DELETE /identity/v3/projects/17805705e6b846d3aa34a351fff8d319 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:10.503155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:10.505552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:10.513990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:10.516122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:10.526336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1123/2267] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:10.527567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:10.580114 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d5433-09b0-49e0-a273-c4c6ccdf05a9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:10.585607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:10.586322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:10.586601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:10.586883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] PATH_INFO: `/v3/users/5723409f99c946c181dc9e8de52cd4d8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:10.588157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5723409f99c946c181dc9e8de52cd4d8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:10.590542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:10.665137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5723409f99c946c181dc9e8de52cd4d8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.682298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5723409f99c946c181dc9e8de52cd4d8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.704611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5723409f99c946c181dc9e8de52cd4d8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.707259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5723409f99c946c181dc9e8de52cd4d8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.707921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5a8961b2-5501-4a8c-8c0f-b7b3acd6bde3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5723409f99c946c181dc9e8de52cd4d8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:10.727960 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1145/2268] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:10 2022] DELETE /identity/v3/users/5723409f99c946c181dc9e8de52cd4d8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:11.080413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.095676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27672454-49c6-4094-b337-12a41cbae6fc service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.186186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.187842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.188465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.188744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] PATH_INFO: `/v3/projects/59518401f11d4444abf7b3dfb5609e03` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.190234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59518401f11d4444abf7b3dfb5609e03)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.192609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.268422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59518401f11d4444abf7b3dfb5609e03', 'request_id': 'req-b517e55f-4417-45f0-a31b-de823ffe9421'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:11.293134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59518401f11d4444abf7b3dfb5609e03', 'request_id': 'req-b517e55f-4417-45f0-a31b-de823ffe9421'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:11.302343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59518401f11d4444abf7b3dfb5609e03', 'request_id': 'req-b517e55f-4417-45f0-a31b-de823ffe9421'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:11.320210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] The token cache is being invalidate because project 59518401f11d4444abf7b3dfb5609e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:11.320210 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:11.321026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:11.322710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1124/2269] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:11 2022] DELETE /identity/v3/projects/59518401f11d4444abf7b3dfb5609e03 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:11.328070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943668a6-738d-4485-899d-56dcb27edefa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.328356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943668a6-738d-4485-899d-56dcb27edefa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.329027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943668a6-738d-4485-899d-56dcb27edefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.355134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-943668a6-738d-4485-899d-56dcb27edefa None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:11.356771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-943668a6-738d-4485-899d-56dcb27edefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:11.403725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.405476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:11.443059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:11.453119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1146/2270] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:11.454707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.455250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.455596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.488347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:11.489799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:11.492051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517e55f-4417-45f0-a31b-de823ffe9421 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.496666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.497617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.497962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.498304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.500696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1147/2271] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:11.504311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:11.506098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.517162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc8e50-00b5-40fb-8ba6-831cec905d37 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.523075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.524130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.524614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.525087 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.525974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:11.526472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.528895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.544481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1148/2272] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:00:11.549163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.563600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57527de-5bbf-4e4b-896a-dbd82b5a5135 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.569529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.570159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.570440 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.570710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.571405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.575314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.589566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:11.592322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.595848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.607552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1125/2273] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:00:11.626343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.632281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1149/2274] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:11 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:11.640697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c58d8e-90f9-4628-94f2-7f1ec4d2ebc3 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.717463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.719057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.719572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.720043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.721048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.723576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.754193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1126/2275] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:11 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:11.758634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.769319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46874f40-c19a-4bf3-b867-1321994b0a83 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.777416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.778243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.778592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.778917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.780107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.783036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.794469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1150/2276] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:11.799120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.810034 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c04ac-d70f-4f86-a08d-fd1f28c949e5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.814947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c5eac-13e6-4f54-b394-6716cb39f8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.815595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c5eac-13e6-4f54-b394-6716cb39f8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.815879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c5eac-13e6-4f54-b394-6716cb39f8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.816152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c5eac-13e6-4f54-b394-6716cb39f8c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.816798 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c5eac-13e6-4f54-b394-6716cb39f8c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.818438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c5eac-13e6-4f54-b394-6716cb39f8c1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.832204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1127/2277] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:11.834779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:11.846712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cec7e-f89e-4bf9-8443-4e56bac239c5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:11.851486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:11.852145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.852421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.852689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] PATH_INFO: `/v3/projects/ab64b5f210fa48d685d039289a1802e7/users/86240c789afe489d8e9d4bec672815f9/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.869475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab64b5f210fa48d685d039289a1802e7, user_id=86240c789afe489d8e9d4bec672815f9, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:11.872172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:11.887388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1151/2278] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:11 2022] PUT /identity/v3/projects/ab64b5f210fa48d685d039289a1802e7/users/86240c789afe489d8e9d4bec672815f9/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:11.898043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.898946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.898946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.944608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] MFA Rules not processed for user `86240c789afe489d8e9d4bec672815f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:11.946546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:12.025651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:12.071681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1128/2279] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:12.076235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.167745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7decc64-140b-4f12-afc3-aedac6b62334 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.172713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.173354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.173634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.173905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.174566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:12.176551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:12.205579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1152/2280] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:12 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:12.210150 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.225683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51534b0b-ec56-4875-9d54-3a9e7069c392 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.290010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.290633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.290911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.291184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.291851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:12.293816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:12.321516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1129/2281] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:12 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:12.325631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.337430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09c42f-62cb-46a4-b280-732b967a845b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.342324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.343574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.343924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.344269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.345060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:12.346748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:12.357677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1153/2282] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:12.361894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.372598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae9769-b2ad-4743-816a-4f1df2c96cbe admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.379628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.380292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.380616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.381202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] PATH_INFO: `/v3/projects/ec72b0c2ad174f52b92b134c712ca543/users/b372e19749b84813912581e1e238b822/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.398838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec72b0c2ad174f52b92b134c712ca543, user_id=b372e19749b84813912581e1e238b822, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:12.401587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:12.418500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1130/2283] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:12 2022] PUT /identity/v3/projects/ec72b0c2ad174f52b92b134c712ca543/users/b372e19749b84813912581e1e238b822/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:12.422479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.441988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddecb8-508b-4c30-8997-5b7557a6569d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.516385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59780531-8ece-4e12-8a67-933aa59ddb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.517306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59780531-8ece-4e12-8a67-933aa59ddb4e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.517656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59780531-8ece-4e12-8a67-933aa59ddb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.517851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59780531-8ece-4e12-8a67-933aa59ddb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.518819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59780531-8ece-4e12-8a67-933aa59ddb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:12.520637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59780531-8ece-4e12-8a67-933aa59ddb4e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:12.533828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1154/2284] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:12.537642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.551038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13adc2-2dba-4637-b8d0-ce0caf8a78ef admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.561235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.562444 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.562726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.563003 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] PATH_INFO: `/v3/projects/ec72b0c2ad174f52b92b134c712ca543/users/b372e19749b84813912581e1e238b822/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.565092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec72b0c2ad174f52b92b134c712ca543, user_id=b372e19749b84813912581e1e238b822, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:12.567876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:12.597759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1131/2285] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:12 2022] PUT /identity/v3/projects/ec72b0c2ad174f52b92b134c712ca543/users/b372e19749b84813912581e1e238b822/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:12.610459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.611132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.611308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.650155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] MFA Rules not processed for user `b372e19749b84813912581e1e238b822`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:12.651418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:12.665871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:12.769874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1155/2286] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:12.783369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:12.786363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:12.886241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc557e8-73cb-4608-863c-47bd1f19ec7d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:12.893732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:12.894011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:12.894119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:12.894362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:12.985627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:13.009293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:13.012155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:13.034268 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1132/2287] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:13.726189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:13.728704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:13.840258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85629355-c60b-46b3-9865-8be615fabac2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:13.845797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:13.846438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:13.846758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:13.847048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] PATH_INFO: `/v3/users/48f744a11c594df98d1cbf5fa265d3c2` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:13.848269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f744a11c594df98d1cbf5fa265d3c2)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:13.850241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:13.908843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f744a11c594df98d1cbf5fa265d3c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:13.917109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f744a11c594df98d1cbf5fa265d3c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:13.930157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f744a11c594df98d1cbf5fa265d3c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:13.930842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f744a11c594df98d1cbf5fa265d3c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:13.931463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-26c2c966-df51-44d5-9073-d485ab8a1f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f744a11c594df98d1cbf5fa265d3c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:13.944709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1156/2288] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:13 2022] DELETE /identity/v3/users/48f744a11c594df98d1cbf5fa265d3c2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:14.113148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.126864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51136a3-07a9-4303-81b4-a20af62026a3 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.195375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.196001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.196280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.196554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] PATH_INFO: `/v3/projects/3e3eb7b7c1be4710b1cdea1e2dc4bf91` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.197694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3eb7b7c1be4710b1cdea1e2dc4bf91)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.199727 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.254252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3eb7b7c1be4710b1cdea1e2dc4bf91', 'request_id': 'req-889a3d02-714a-4f4b-952d-4b307fc73eaf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:14.262385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3eb7b7c1be4710b1cdea1e2dc4bf91', 'request_id': 'req-889a3d02-714a-4f4b-952d-4b307fc73eaf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:14.268287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3eb7b7c1be4710b1cdea1e2dc4bf91', 'request_id': 'req-889a3d02-714a-4f4b-952d-4b307fc73eaf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:14.279498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] The token cache is being invalidate because project 3e3eb7b7c1be4710b1cdea1e2dc4bf91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:14.279782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:14.280448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-889a3d02-714a-4f4b-952d-4b307fc73eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:14.281914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1133/2289] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:14 2022] DELETE /identity/v3/projects/3e3eb7b7c1be4710b1cdea1e2dc4bf91 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:14.286869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.287158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.287478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.308105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:14.309406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:14.336980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:14.388605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1157/2290] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:14.396734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.397043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.397298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.417363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:14.418586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:14.427392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1134/2291] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:14.431815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.442295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e8f65-90f8-4d05-b5aa-c59116719155 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.447800 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.448521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.448801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.449172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.449900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:14.450206 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.452248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.464468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1158/2292] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:14.468974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.478502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe35b64-5dc7-4093-8393-9b26dfe59afb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.486425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.487035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.487486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.488518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.488518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.490322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.521710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1135/2293] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:14.525850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.535849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcf2f-d5c8-48a6-b706-05f347c16fd4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.598506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.599128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.599447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.599726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.600357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.602268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.626907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1159/2294] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:14.631256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.641872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7e634-c77f-4636-9f79-3f0fd50c4b7d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.646739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.647393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.647679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.647953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.648597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.650198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.661248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1136/2295] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:14.665136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.676875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd029e-e63e-4668-b1d7-bf5970c5f893 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.681703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0279-372a-418b-88df-a4ee14ef29ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.682383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0279-372a-418b-88df-a4ee14ef29ec admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.682665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0279-372a-418b-88df-a4ee14ef29ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.682941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0279-372a-418b-88df-a4ee14ef29ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.683632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e0279-372a-418b-88df-a4ee14ef29ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.685429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e0279-372a-418b-88df-a4ee14ef29ec admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.696737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1160/2296] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:14.701128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.711145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481b411-316d-4af3-8bb4-c1d3ad44d9ae admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.715951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.716795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.717110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.717464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] PATH_INFO: `/v3/projects/1b624c67342547b692ae00d9413c0376/users/468370ded65348e5849c3fe148c01ad0/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.734207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b624c67342547b692ae00d9413c0376, user_id=468370ded65348e5849c3fe148c01ad0, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.737095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.751970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1137/2297] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:14 2022] PUT /identity/v3/projects/1b624c67342547b692ae00d9413c0376/users/468370ded65348e5849c3fe148c01ad0/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:14.763618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.764084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.764216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.791884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] MFA Rules not processed for user `468370ded65348e5849c3fe148c01ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:14.793089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:14.861214 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1161/2298] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:14.865635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.883259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:14.933852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09109dec-8a4e-4101-8698-857dea7482ed admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:14.939023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:14.939698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:14.940000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:14.940280 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:14.940918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:14.942857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:14.974558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1138/2299] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:14.975980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:14.990009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbea22a-2053-4166-ac14-1fb33f1ff6fc None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:15.017648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:15.058678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:15.059324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.059625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.059886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.060523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:15.062511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:15.089234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1162/2300] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:14 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:15.090363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:15.101525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3a2f5-d078-41d4-a2c8-0cf944da74ed admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:15.115471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:15.115471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.115471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.115471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.116339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:15.116339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:15.122918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1139/2301] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:15.127153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:15.137566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c347090-36dd-492e-9c49-c06b33cd6b2d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:15.142660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:15.143341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.143621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.143891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] PATH_INFO: `/v3/projects/3f1960110c334716b1afde7334b572b7/users/a862ea1374ca4b54ac18330a33222d53/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.159032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1960110c334716b1afde7334b572b7, user_id=a862ea1374ca4b54ac18330a33222d53, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:15.161740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:15.178336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1163/2302] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:15 2022] PUT /identity/v3/projects/3f1960110c334716b1afde7334b572b7/users/a862ea1374ca4b54ac18330a33222d53/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:15.182383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:15.192441 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0453a5-9484-45ce-a509-9b1501405c42 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:15.250284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c026d02-f81c-45bf-b5d3-1429e144df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:15.251024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c026d02-f81c-45bf-b5d3-1429e144df08 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.251519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c026d02-f81c-45bf-b5d3-1429e144df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.251856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c026d02-f81c-45bf-b5d3-1429e144df08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.252626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c026d02-f81c-45bf-b5d3-1429e144df08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:15.254546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c026d02-f81c-45bf-b5d3-1429e144df08 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:15.265538 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1140/2303] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:15.269882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:15.280682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8811794-fb00-43cd-aaf1-4e80861b59ee admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:15.285510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:15.286177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.286460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.286732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] PATH_INFO: `/v3/projects/3f1960110c334716b1afde7334b572b7/users/a862ea1374ca4b54ac18330a33222d53/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.289133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1960110c334716b1afde7334b572b7, user_id=a862ea1374ca4b54ac18330a33222d53, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:15.291986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:15.304634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1164/2304] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:15 2022] PUT /identity/v3/projects/3f1960110c334716b1afde7334b572b7/users/a862ea1374ca4b54ac18330a33222d53/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:15.316029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.316332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.316762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.353962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] MFA Rules not processed for user `a862ea1374ca4b54ac18330a33222d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:15.355129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:15.427176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1141/2305] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:15.439166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:15.440919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:15.507026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf21ec-2ef4-4c61-83dd-0b549b4880f3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:15.516983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb23e9-7414-42ea-973b-5e35e498674b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:15.516983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb23e9-7414-42ea-973b-5e35e498674b service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:15.516983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb23e9-7414-42ea-973b-5e35e498674b service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:15.516983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb23e9-7414-42ea-973b-5e35e498674b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:15.525653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb23e9-7414-42ea-973b-5e35e498674b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:15.527108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb23e9-7414-42ea-973b-5e35e498674b service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:15.539106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1165/2306] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:15.968464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:15.981341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:16.046501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b023f7b-6597-4189-8daf-c09d6b20f815 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:16.052740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:16.053456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:16.053844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:16.054469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] PATH_INFO: `/v3/users/468370ded65348e5849c3fe148c01ad0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:16.056383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=468370ded65348e5849c3fe148c01ad0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:16.058696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:16.112039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468370ded65348e5849c3fe148c01ad0'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.120334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468370ded65348e5849c3fe148c01ad0'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.121071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468370ded65348e5849c3fe148c01ad0'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.141255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468370ded65348e5849c3fe148c01ad0'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.151088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468370ded65348e5849c3fe148c01ad0'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.159678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1142/2307] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:15 2022] DELETE /identity/v3/users/468370ded65348e5849c3fe148c01ad0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:16.164627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:16.165538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:16.165538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:16.191559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:16.193035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:16.283899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1166/2308] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:16.295161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:16.296799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:16.366783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967e6d5-1894-4a02-89b5-b904ebb905dd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:16.370770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:16.371520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:16.371850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:16.372129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:16.386360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:16.387993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:16.398679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1143/2309] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:16.679139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:16.680728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:16.693777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8281-b151-4628-8daa-6b23dcb51908 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:16.699035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:16.699711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:16.700017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:16.700282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] PATH_INFO: `/v3/projects/1b624c67342547b692ae00d9413c0376` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:16.701403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b624c67342547b692ae00d9413c0376)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:16.703360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:16.720317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:16.754631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b624c67342547b692ae00d9413c0376', 'request_id': 'req-0310498e-2ad3-4826-9191-4272529d8c97'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.761936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b624c67342547b692ae00d9413c0376', 'request_id': 'req-0310498e-2ad3-4826-9191-4272529d8c97'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.767775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b624c67342547b692ae00d9413c0376', 'request_id': 'req-0310498e-2ad3-4826-9191-4272529d8c97'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.778698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] The token cache is being invalidate because project 1b624c67342547b692ae00d9413c0376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:16.778986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.779659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.781177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1167/2310] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:16 2022] DELETE /identity/v3/projects/1b624c67342547b692ae00d9413c0376 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:16.785368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:16.797178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98af95-9408-4def-8b7c-9b4448c27c82 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:16.827814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:16.855993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:16.856626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:16.856907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:16.857181 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] PATH_INFO: `/v3/users/a862ea1374ca4b54ac18330a33222d53` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:16.859075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a862ea1374ca4b54ac18330a33222d53)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:16.861063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:16.918183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a862ea1374ca4b54ac18330a33222d53'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.927519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a862ea1374ca4b54ac18330a33222d53'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.928153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a862ea1374ca4b54ac18330a33222d53'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.942028 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a862ea1374ca4b54ac18330a33222d53'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.945530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:16.946937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:16.956117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-248d1177-0547-4fb6-8e03-b7501a903a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a862ea1374ca4b54ac18330a33222d53'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:16.961437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310498e-2ad3-4826-9191-4272529d8c97 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:16.962712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1144/2311] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:16 2022] DELETE /identity/v3/users/a862ea1374ca4b54ac18330a33222d53 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:17.031757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.032632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.032923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.033199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] PATH_INFO: `/v3/users/8dfef6201aef4087b0dfb9751fe4ea0e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.034673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfef6201aef4087b0dfb9751fe4ea0e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.038018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.087276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfef6201aef4087b0dfb9751fe4ea0e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.095775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfef6201aef4087b0dfb9751fe4ea0e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.112409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfef6201aef4087b0dfb9751fe4ea0e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.113312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfef6201aef4087b0dfb9751fe4ea0e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.113933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfef6201aef4087b0dfb9751fe4ea0e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.128131 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1168/2312] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:16 2022] DELETE /identity/v3/users/8dfef6201aef4087b0dfb9751fe4ea0e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:17.132737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.133193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.133298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.153312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:17.154505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.226710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1145/2313] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:17.237179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.238659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.274198 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:17.280314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.283029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.297458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a93c9-469e-43f0-bb32-5fca3e6637df None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.304063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.304764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.304985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.305267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] PATH_INFO: `/v3/projects/3f1960110c334716b1afde7334b572b7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.306477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1960110c334716b1afde7334b572b7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.309172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.320143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5125de-2176-4553-bae7-982d39653593 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.324337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe82b72-ef42-4e7c-a526-a1f02fc123f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.325094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82b72-ef42-4e7c-a526-a1f02fc123f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.325438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82b72-ef42-4e7c-a526-a1f02fc123f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.325817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe82b72-ef42-4e7c-a526-a1f02fc123f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.338274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe82b72-ef42-4e7c-a526-a1f02fc123f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.339759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe82b72-ef42-4e7c-a526-a1f02fc123f9 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.352083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1169/2314] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:17.376429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:17.378494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1960110c334716b1afde7334b572b7', 'request_id': 'req-d334e0a3-4887-4d91-99f2-4b7620f93b02'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.391584 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1960110c334716b1afde7334b572b7', 'request_id': 'req-d334e0a3-4887-4d91-99f2-4b7620f93b02'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.399748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1960110c334716b1afde7334b572b7', 'request_id': 'req-d334e0a3-4887-4d91-99f2-4b7620f93b02'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.412735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] The token cache is being invalidate because project 3f1960110c334716b1afde7334b572b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:17.413036 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.413816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d334e0a3-4887-4d91-99f2-4b7620f93b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.415820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1146/2315] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:17 2022] DELETE /identity/v3/projects/3f1960110c334716b1afde7334b572b7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:17.421104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.421566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.421846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.450148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:17.451416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.562435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1170/2316] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:17.570122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.570419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.570696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.592101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:17.594040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.602362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1147/2317] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:17.606608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.616182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ab657-e52d-449d-b3c5-24b12280370f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.621172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.621703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.621980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.622247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.622904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:17.623222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.625626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.638530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1171/2318] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:00:17.643556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.658759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac27e6a-a229-4027-a9ab-35b3d8a1704f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.663778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.664421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.664704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.665006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.666089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.668317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.697914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1148/2319] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:17 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:17.702029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.712824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78534eb5-1e2d-4eb2-a61b-12680ffeac50 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.768873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.770924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.775626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.776299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.776570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.776834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.777480 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.779456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.789067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd87ff-40be-467f-b78f-365c4d6d0fde admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.796821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.797573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.797868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.798183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] PATH_INFO: `/v3/projects/60280bb8b9a647d18d184a06b0b60612` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.799371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60280bb8b9a647d18d184a06b0b60612)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.801487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.808155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1172/2320] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:17 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:17.812637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.823634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1ae8c-3d99-486f-9e99-61e0a5a6048a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.829611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.830108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.830417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.830693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.831430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.833468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.850180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1173/2321] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:17.851229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.861388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a8052-0988-47b8-9c92-109715f8222c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.864539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60280bb8b9a647d18d184a06b0b60612', 'request_id': 'req-8f2987d6-e103-4446-90bc-24667af72813'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.873096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60280bb8b9a647d18d184a06b0b60612', 'request_id': 'req-8f2987d6-e103-4446-90bc-24667af72813'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.879478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60280bb8b9a647d18d184a06b0b60612', 'request_id': 'req-8f2987d6-e103-4446-90bc-24667af72813'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.896578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:17.902564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] The token cache is being invalidate because project 60280bb8b9a647d18d184a06b0b60612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:17.902957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.903737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:17.905336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1149/2322] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:17 2022] DELETE /identity/v3/projects/60280bb8b9a647d18d184a06b0b60612 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:17.909409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.910708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.931128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.931785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.932153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.932368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.933016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.934618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.949098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1174/2323] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:17.950135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:17.952745 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:17.972148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02db90-41e0-4a84-9efb-b02ad7d7e3d6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.977459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bdc23-59fb-4b22-a06d-1dc201c72fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.978121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bdc23-59fb-4b22-a06d-1dc201c72fea admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.978235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2987d6-e103-4446-90bc-24667af72813 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:17.978534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bdc23-59fb-4b22-a06d-1dc201c72fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.978815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bdc23-59fb-4b22-a06d-1dc201c72fea admin admin] PATH_INFO: `/v3/projects/068bd7ab009642509e95dd5cdb40e96b/users/b38833db5074484a9850624a3b1aa481/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.982770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:17.983922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:17.984195 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:17.984465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] PATH_INFO: `/v3/users/9d31bb74ad2945dc8ba631d8be316179` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:17.985595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d31bb74ad2945dc8ba631d8be316179)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.987575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:17.994314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6bdc23-59fb-4b22-a06d-1dc201c72fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=068bd7ab009642509e95dd5cdb40e96b, user_id=b38833db5074484a9850624a3b1aa481, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:17.997140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6bdc23-59fb-4b22-a06d-1dc201c72fea admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.012329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1175/2324] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:17 2022] PUT /identity/v3/projects/068bd7ab009642509e95dd5cdb40e96b/users/b38833db5074484a9850624a3b1aa481/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:18.028000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.028454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.028909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.040076 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:18.054977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d31bb74ad2945dc8ba631d8be316179'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.068013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d31bb74ad2945dc8ba631d8be316179'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.068864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d31bb74ad2945dc8ba631d8be316179'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.080639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] MFA Rules not processed for user `b38833db5074484a9850624a3b1aa481`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:18.081850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:18.082005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d31bb74ad2945dc8ba631d8be316179'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.095951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d31bb74ad2945dc8ba631d8be316179'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.107492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1150/2325] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:17 2022] DELETE /identity/v3/users/9d31bb74ad2945dc8ba631d8be316179 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:18.174361 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.175402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1176/2326] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:18.176596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:18.265311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e9fff-25f0-4d89-ac91-b36744b1c40e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.271077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.271838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.272156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.272431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.273101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.281806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.323278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1151/2327] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:18 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:18.325307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.415733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.419274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:18.455747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576741-ce3b-47e1-a585-8f06bd4cc962 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.520527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:18.552813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.553688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.554030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.554367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.555084 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.557446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.562754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1372a-444b-4ae6-bc3f-97c4ab86ceba admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.568509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.569132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.569419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.569695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] PATH_INFO: `/v3/projects/bd79e937afca45aca6dba157ea05c944` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.570854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd79e937afca45aca6dba157ea05c944)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.573241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.589760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1177/2328] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:18 2022] POST /identity/v3/users => generated 291 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:18.594462 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.615222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc87472-fe5a-48a3-a7a1-2f394e91b783 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.619977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.620637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.620918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.621186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.621891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.623578 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.627278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:18.634082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1178/2329] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:18.638746 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.642211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd79e937afca45aca6dba157ea05c944', 'request_id': 'req-2c01b912-3c41-42ae-9053-b985e06394f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.650585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28455318-4544-434d-bda9-79879d2b8e38 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.651992 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd79e937afca45aca6dba157ea05c944', 'request_id': 'req-2c01b912-3c41-42ae-9053-b985e06394f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.665433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd79e937afca45aca6dba157ea05c944', 'request_id': 'req-2c01b912-3c41-42ae-9053-b985e06394f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.693525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] The token cache is being invalidate because project bd79e937afca45aca6dba157ea05c944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:18.693869 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.695390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:18.697528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1152/2330] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:18 2022] DELETE /identity/v3/projects/bd79e937afca45aca6dba157ea05c944 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:18.702062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.704780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:18.729211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.729791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.730160 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.730502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] PATH_INFO: `/v3/projects/40e946ea420946c388ecb8b487419d10/users/97fc06fa095c4960b15c927fb568615c/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.755608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e946ea420946c388ecb8b487419d10, user_id=97fc06fa095c4960b15c927fb568615c, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.758713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.777648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1179/2331] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:18 2022] PUT /identity/v3/projects/40e946ea420946c388ecb8b487419d10/users/97fc06fa095c4960b15c927fb568615c/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:18.783863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.787080 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:18.809429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01b912-3c41-42ae-9053-b985e06394f1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.893307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cd387-fbf2-4e14-a1a7-30810ed664b7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.898419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.899510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.899951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.900493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.901149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.902783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.914340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.915651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.916328 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.916895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] PATH_INFO: `/v3/users/9c5a813e4833422da61dbe928e420e89` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.918464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5a813e4833422da61dbe928e420e89)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.919338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1180/2332] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:18.920102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:18.922338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.932464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238534-0317-471e-849e-01d7602f0655 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:18.938453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3137ee-53d1-434a-878a-75299a802af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:18.939132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3137ee-53d1-434a-878a-75299a802af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.939528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3137ee-53d1-434a-878a-75299a802af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.939807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3137ee-53d1-434a-878a-75299a802af5 admin admin] PATH_INFO: `/v3/projects/40e946ea420946c388ecb8b487419d10/users/97fc06fa095c4960b15c927fb568615c/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.942515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3137ee-53d1-434a-878a-75299a802af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e946ea420946c388ecb8b487419d10, user_id=97fc06fa095c4960b15c927fb568615c, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:18.946076 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3137ee-53d1-434a-878a-75299a802af5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:18.961754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1181/2333] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:18 2022] PUT /identity/v3/projects/40e946ea420946c388ecb8b487419d10/users/97fc06fa095c4960b15c927fb568615c/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:18.973456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:18.973853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:18.974472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:18.988858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5a813e4833422da61dbe928e420e89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.000943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5a813e4833422da61dbe928e420e89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.002075 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5a813e4833422da61dbe928e420e89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.012189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] MFA Rules not processed for user `97fc06fa095c4960b15c927fb568615c`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:19.013557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.017179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5a813e4833422da61dbe928e420e89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.019454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:19.026522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5a813e4833422da61dbe928e420e89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.027714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:19.033746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1153/2334] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:18 2022] DELETE /identity/v3/users/9c5a813e4833422da61dbe928e420e89 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:19.091608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1182/2335] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:19.104778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.106458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.172243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569938-777e-4408-b3e3-4f90d20af76e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.176826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.177973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.178491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.179352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.249744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.251187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.261575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1154/2336] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:19.317576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.319028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.332883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.334427 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.384313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc210fa-f65a-4037-9cad-f3359098ee04 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.388249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.388895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.389166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.390162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.391890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.421203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbe792-e1b3-427f-9894-f0523f45d49f service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.425991 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.426714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.427181 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.427560 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] PATH_INFO: `/v3/projects/887b6c9a7444430388e541b985d53035` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.428753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887b6c9a7444430388e541b985d53035)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.430793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.452032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.454122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c6904-30e7-4722-84b8-4ed146b999b6 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.463495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1183/2337] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:19.487567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:19.492581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887b6c9a7444430388e541b985d53035', 'request_id': 'req-95ab0e4c-d475-4639-a95f-9f16bd99cd63'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.501038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887b6c9a7444430388e541b985d53035', 'request_id': 'req-95ab0e4c-d475-4639-a95f-9f16bd99cd63'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.507705 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887b6c9a7444430388e541b985d53035', 'request_id': 'req-95ab0e4c-d475-4639-a95f-9f16bd99cd63'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.521489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] The token cache is being invalidate because project 887b6c9a7444430388e541b985d53035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:19.521786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.522449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95ab0e4c-d475-4639-a95f-9f16bd99cd63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:19.523997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1155/2338] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:19 2022] DELETE /identity/v3/projects/887b6c9a7444430388e541b985d53035 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:19.528956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.529254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.529534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.552629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:19.554613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.628891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1184/2339] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:19.630424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.630760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.631046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.653038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:19.654601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:19.663806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1156/2340] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:19.668189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.690387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de1bc-5c29-4864-b97d-4ac507cbb2aa None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.696075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.696992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.697448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.698697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.699453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:19.699766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.701478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.704227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:19.716190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1185/2341] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:19.720723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.730214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b897-a0ea-4037-bbef-cc75e8d1a043 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.736253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.737367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.737656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.737951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.738712 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.740763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.779242 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1157/2342] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:19 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:19.782656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.794177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751c64f-b742-46a9-be55-46d90f494bca admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.865379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.867056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.867056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.867056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.867964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.869678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.901319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1186/2343] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:19 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:19.902554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.913541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f382e-f2b5-4f01-8398-39160ba5137d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.918134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.918735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.919013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.919374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.919940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.922153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.932586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1158/2344] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:19.936864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.947506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b96450-9eae-4f0a-9d96-4fb18f269b4c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.953461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862615b1-f44e-4040-9074-a485c5f96e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.954487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862615b1-f44e-4040-9074-a485c5f96e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.954761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862615b1-f44e-4040-9074-a485c5f96e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.955147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862615b1-f44e-4040-9074-a485c5f96e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:19.955903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862615b1-f44e-4040-9074-a485c5f96e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:19.957611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862615b1-f44e-4040-9074-a485c5f96e96 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:19.972203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1187/2345] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:19.976522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:19.988940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c9d8-e7ee-4e7c-b888-00dafd600bee admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:19.994534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:19.995227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:19.995512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:19.996030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] PATH_INFO: `/v3/projects/a7dc539727694ed1b3d70900860673bd/users/6aa62784a6fa45b4b37b8af70ab16e31/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:20.019730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7dc539727694ed1b3d70900860673bd, user_id=6aa62784a6fa45b4b37b8af70ab16e31, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:20.022699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:20.038780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1159/2346] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:19 2022] PUT /identity/v3/projects/a7dc539727694ed1b3d70900860673bd/users/6aa62784a6fa45b4b37b8af70ab16e31/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:20.050861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:20.051148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:20.051573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:20.088155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] MFA Rules not processed for user `6aa62784a6fa45b4b37b8af70ab16e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:20.089766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:20.090632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:20.093272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:20.153476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c7cb4-c36c-442a-9f36-3779776b2d49 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:20.158412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:20.159035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:20.159348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:20.159620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:20.161887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:20.164488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1188/2347] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:20.178244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:20.180120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:20.196681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd5e98-6057-4e42-956a-01be6db9c90d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:20.203244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846612ee-b44e-4015-a7d8-3999fb9a07a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:20.204826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846612ee-b44e-4015-a7d8-3999fb9a07a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:20.205652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846612ee-b44e-4015-a7d8-3999fb9a07a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:20.206274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846612ee-b44e-4015-a7d8-3999fb9a07a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:20.222099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846612ee-b44e-4015-a7d8-3999fb9a07a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:20.223607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846612ee-b44e-4015-a7d8-3999fb9a07a7 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:20.233138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1189/2348] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:00:20.245097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:20.246588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:20.256572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1160/2349] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:00:22.720451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664928d-cedf-470b-9c60-99127a46d1d6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:22.720910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664928d-cedf-470b-9c60-99127a46d1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:22.720910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664928d-cedf-470b-9c60-99127a46d1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:22.743673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5664928d-cedf-470b-9c60-99127a46d1d6 None None] MFA Rules not processed for user `6aa62784a6fa45b4b37b8af70ab16e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:22.744881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5664928d-cedf-470b-9c60-99127a46d1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:22.752476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1190/2350] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:22.936428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:22.937910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:22.963848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:23.031987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f129c-b05a-47c7-bda3-27988f435f82 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:23.043588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:23.044385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:23.044669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:23.044941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] PATH_INFO: `/v3/users/b38833db5074484a9850624a3b1aa481` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:23.046382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38833db5074484a9850624a3b1aa481)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:23.054368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:23.120127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38833db5074484a9850624a3b1aa481'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.131987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38833db5074484a9850624a3b1aa481'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.132702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38833db5074484a9850624a3b1aa481'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.148700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38833db5074484a9850624a3b1aa481'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.160453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38833db5074484a9850624a3b1aa481'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.167995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1161/2351] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:22 2022] DELETE /identity/v3/users/b38833db5074484a9850624a3b1aa481 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:23.172258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:23.172625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:23.172806 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:23.201384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:23.202601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:23.247047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:23.274932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1191/2352] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:23.288073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:23.289634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:23.407945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016b80f-d597-4d80-b62f-5d4f085222b3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:23.412485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:23.413386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:23.413673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:23.414040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:23.430769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:23.432409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:23.442948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1162/2353] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:23.769772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:23.771657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:23.820720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537ea7f-8875-4cc8-88cb-6ad0c5ad0c1c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:23.834153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:23.835170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:23.835574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:23.835923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] PATH_INFO: `/v3/projects/068bd7ab009642509e95dd5cdb40e96b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:23.837065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=068bd7ab009642509e95dd5cdb40e96b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:23.839156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:23.915506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '068bd7ab009642509e95dd5cdb40e96b', 'request_id': 'req-608c870a-00d8-4695-b0ca-92dd59512d32'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.926330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '068bd7ab009642509e95dd5cdb40e96b', 'request_id': 'req-608c870a-00d8-4695-b0ca-92dd59512d32'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.933832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '068bd7ab009642509e95dd5cdb40e96b', 'request_id': 'req-608c870a-00d8-4695-b0ca-92dd59512d32'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.951317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] The token cache is being invalidate because project 068bd7ab009642509e95dd5cdb40e96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:23.951945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.953955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:23.959260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1192/2354] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:23 2022] DELETE /identity/v3/projects/068bd7ab009642509e95dd5cdb40e96b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:23.960996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:23.973528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652ab4c-00e4-4ce5-9d1f-94b68bf51f92 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:24.020640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:24.044556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:24.045271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.045561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.045843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] PATH_INFO: `/v3/users/97fc06fa095c4960b15c927fb568615c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.047967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fc06fa095c4960b15c927fb568615c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:24.052925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:24.134253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fc06fa095c4960b15c927fb568615c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.147254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fc06fa095c4960b15c927fb568615c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.148502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fc06fa095c4960b15c927fb568615c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.163262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fc06fa095c4960b15c927fb568615c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.171418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-830a4cfe-ffd8-4f66-95ee-95b7679697b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fc06fa095c4960b15c927fb568615c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.178286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1163/2355] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:23 2022] DELETE /identity/v3/users/97fc06fa095c4960b15c927fb568615c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:24.481269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:24.482989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:24.498325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c870a-00d8-4695-b0ca-92dd59512d32 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:24.580819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:24.581393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.581670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.581941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] PATH_INFO: `/v3/projects/40e946ea420946c388ecb8b487419d10` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.583144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e946ea420946c388ecb8b487419d10)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:24.585081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:24.636505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e946ea420946c388ecb8b487419d10', 'request_id': 'req-f6a759e2-aad3-46ea-8757-2d69f1cceefe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.644831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e946ea420946c388ecb8b487419d10', 'request_id': 'req-f6a759e2-aad3-46ea-8757-2d69f1cceefe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.650752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e946ea420946c388ecb8b487419d10', 'request_id': 'req-f6a759e2-aad3-46ea-8757-2d69f1cceefe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.652626 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:24.665774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] The token cache is being invalidate because project 40e946ea420946c388ecb8b487419d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:24.666341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.667413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f6a759e2-aad3-46ea-8757-2d69f1cceefe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:24.668955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1193/2356] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:24 2022] DELETE /identity/v3/projects/40e946ea420946c388ecb8b487419d10 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:24.674597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.674935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.675235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.696026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:24.697275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:24.758816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1164/2357] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:24.766839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.767367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.767653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.788402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:24.789635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:24.798552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1194/2358] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:24.802657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:24.813845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108f39-2084-46ea-b643-85c4fc3bb9bd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:24.818575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:24.819263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.819547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.819887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.820615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:24.820924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:24.822590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:24.839525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1165/2359] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:24.840985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:24.852042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebc1e3-c535-4ab0-9445-88e1b2dacf18 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:24.857038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:24.857905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.858189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.858622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.859819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:24.861945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:24.893943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1195/2360] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:24 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:24.899104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:24.913096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a54c-acf6-4f4d-b74b-af23c3dde511 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:24.974101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:24.993627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:24.994255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.994524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.994801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.995485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:24.997420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.026975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1166/2361] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:24 2022] POST /identity/v3/users => generated 305 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:25.027896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.040271 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937817-927e-420a-9720-0717d07853b7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.046909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.047930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.047930 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.048138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.048870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.050544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.061190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1196/2362] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:25.067566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.078495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d326-afb3-4c1c-a7f4-8f62e01b19ba admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.083743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05be25-7f0e-4e2e-a46a-e03ad761c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.084536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05be25-7f0e-4e2e-a46a-e03ad761c42b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.084824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05be25-7f0e-4e2e-a46a-e03ad761c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.085096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05be25-7f0e-4e2e-a46a-e03ad761c42b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.085731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05be25-7f0e-4e2e-a46a-e03ad761c42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.087419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05be25-7f0e-4e2e-a46a-e03ad761c42b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.102005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1167/2363] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:25.106069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.116945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a075da-ec91-44ca-b079-62e7fcd0111f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.123475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.124227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.124518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.124761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] PATH_INFO: `/v3/projects/8cd6a73fa76c49df8bd90d7304876101/users/88d6e03c8d1d45f6827882925217f955/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.144515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd6a73fa76c49df8bd90d7304876101, user_id=88d6e03c8d1d45f6827882925217f955, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.147325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.161127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1197/2364] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:25 2022] PUT /identity/v3/projects/8cd6a73fa76c49df8bd90d7304876101/users/88d6e03c8d1d45f6827882925217f955/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:25.184915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.185202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.185476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.217264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] MFA Rules not processed for user `88d6e03c8d1d45f6827882925217f955`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:25.218985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:25.284843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1168/2365] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:25.289445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.361134 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435594c-5cb3-4878-aef7-f146f322dcfe admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.366117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.366748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.367029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.367337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.368169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.370548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.402983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1198/2366] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:25 2022] POST /identity/v3/projects => generated 407 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:25.407989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.420043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9796f9b-8b5f-49b8-9d09-aecb981bcae4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.499557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.500187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.500467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.500742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.501399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.503391 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.528599 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1169/2367] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:25 2022] POST /identity/v3/users => generated 310 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:25.533099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.543752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a61de-8bb0-46b5-b9ef-88f7bb9d1c51 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.548997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.550174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.550174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.550174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.550792 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.552453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.565341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1199/2368] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:25.571559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.585015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e54f8e-52e9-4387-9702-69b506b8161e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.594445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.595305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.595595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.596059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] PATH_INFO: `/v3/projects/2347287e49a140718a7eaa3e71feed43/users/2a4a3da2342d43ce96b007fb8f0896d7/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.628338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2347287e49a140718a7eaa3e71feed43, user_id=2a4a3da2342d43ce96b007fb8f0896d7, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.646438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.672360 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1170/2369] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:25 2022] PUT /identity/v3/projects/2347287e49a140718a7eaa3e71feed43/users/2a4a3da2342d43ce96b007fb8f0896d7/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:25.674590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.688190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f9255-efc0-46eb-a462-8aad2b90802a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.777336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636619f2-c87a-48bd-ae55-e388b2cdc0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.778089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636619f2-c87a-48bd-ae55-e388b2cdc0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.778441 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636619f2-c87a-48bd-ae55-e388b2cdc0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.778772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636619f2-c87a-48bd-ae55-e388b2cdc0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.779587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636619f2-c87a-48bd-ae55-e388b2cdc0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.782391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636619f2-c87a-48bd-ae55-e388b2cdc0a6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.794775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1200/2370] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:25.800057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:25.828170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9bfef-021b-4fcd-826f-0934713a9764 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:25.835588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:25.836333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.836606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.837132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] PATH_INFO: `/v3/projects/2347287e49a140718a7eaa3e71feed43/users/2a4a3da2342d43ce96b007fb8f0896d7/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.841528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2347287e49a140718a7eaa3e71feed43, user_id=2a4a3da2342d43ce96b007fb8f0896d7, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:25.845752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:25.860070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1171/2371] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:25 2022] PUT /identity/v3/projects/2347287e49a140718a7eaa3e71feed43/users/2a4a3da2342d43ce96b007fb8f0896d7/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:25.875259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:25.875740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:25.876079 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:25.884542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:25.919884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] MFA Rules not processed for user `2a4a3da2342d43ce96b007fb8f0896d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:25.921999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:26.012003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1201/2372] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:26.020643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:26.022461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:26.035079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:26.096850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac336893-5ae8-4f51-9061-84ca22fc6c2f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:26.100592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:26.101208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:26.101487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:26.101762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:26.113031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:26.114486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:26.126365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1172/2373] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:26 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:26.322298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:26.324262 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:26.350984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88788a2a-1429-4f94-a7a4-46415963fc36 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:26.355052 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b863c-4c05-4e01-9c90-488ab05418d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:26.355711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b863c-4c05-4e01-9c90-488ab05418d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:26.355990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b863c-4c05-4e01-9c90-488ab05418d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:26.356261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b863c-4c05-4e01-9c90-488ab05418d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:26.429092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b863c-4c05-4e01-9c90-488ab05418d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:26.430381 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b863c-4c05-4e01-9c90-488ab05418d7 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:26.442620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1202/2374] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:26 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:36.120523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=23 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:36.436475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=61 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:42.216651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:42.218089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:42.284165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed060f-5daf-464a-b74f-c39bc5693b3d service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:42.289162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:42.289796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:42.290079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:42.290362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] PATH_INFO: `/v3/users/86240c789afe489d8e9d4bec672815f9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:42.291529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86240c789afe489d8e9d4bec672815f9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:42.293464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:42.347289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86240c789afe489d8e9d4bec672815f9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:42.355009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86240c789afe489d8e9d4bec672815f9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:42.355800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86240c789afe489d8e9d4bec672815f9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:42.368101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86240c789afe489d8e9d4bec672815f9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:42.374779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86240c789afe489d8e9d4bec672815f9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:42.381349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1173/2375] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:42 2022] DELETE /identity/v3/users/86240c789afe489d8e9d4bec672815f9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:42.385970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:42.386474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:42.386583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:42.405956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:42.407165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:42.464835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1203/2376] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:42.474797 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:42.476373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:42.541254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11abc58-af13-4959-8b4a-33ac08eb8ea6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:42.545407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:42.546021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:42.546301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:42.546580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:42.555925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:42.557355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:42.568145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1174/2377] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:42.944344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:42.945863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:42.957887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3337d-9bae-4ef6-9e89-e28c1296821e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:42.962411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:42.963037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:42.963344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:42.963613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] PATH_INFO: `/v3/projects/ab64b5f210fa48d685d039289a1802e7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:42.964744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab64b5f210fa48d685d039289a1802e7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:42.966659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:43.032621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab64b5f210fa48d685d039289a1802e7', 'request_id': 'req-89ce435b-8643-4f43-b59a-10faaf5bd562'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.041394 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab64b5f210fa48d685d039289a1802e7', 'request_id': 'req-89ce435b-8643-4f43-b59a-10faaf5bd562'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.048264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab64b5f210fa48d685d039289a1802e7', 'request_id': 'req-89ce435b-8643-4f43-b59a-10faaf5bd562'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.060246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] The token cache is being invalidate because project ab64b5f210fa48d685d039289a1802e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:43.060674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.061871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.066354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1204/2378] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:42 2022] DELETE /identity/v3/projects/ab64b5f210fa48d685d039289a1802e7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:43.067800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:43.078529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5caed5-6a11-48cc-b84b-e66b61561b18 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:43.100198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:43.143270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:43.143906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:43.144187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:43.144457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] PATH_INFO: `/v3/users/b372e19749b84813912581e1e238b822` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:43.145577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b372e19749b84813912581e1e238b822)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:43.147522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:43.195591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b372e19749b84813912581e1e238b822'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.203851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b372e19749b84813912581e1e238b822'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.204589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b372e19749b84813912581e1e238b822'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.218530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b372e19749b84813912581e1e238b822'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.227026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-32d235ed-45ea-4fa1-a514-6f2de4f57277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b372e19749b84813912581e1e238b822'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.233167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1175/2379] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:43 2022] DELETE /identity/v3/users/b372e19749b84813912581e1e238b822 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:43.512145 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:43.514326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:43.530575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce435b-8643-4f43-b59a-10faaf5bd562 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:43.594722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:43.595395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:43.595679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:43.595955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] PATH_INFO: `/v3/projects/ec72b0c2ad174f52b92b134c712ca543` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:43.597103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec72b0c2ad174f52b92b134c712ca543)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:43.599155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:43.632239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:43.655340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec72b0c2ad174f52b92b134c712ca543', 'request_id': 'req-e5059cf2-41cc-49ce-bbf8-390d741a4608'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.663510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec72b0c2ad174f52b92b134c712ca543', 'request_id': 'req-e5059cf2-41cc-49ce-bbf8-390d741a4608'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.670071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec72b0c2ad174f52b92b134c712ca543', 'request_id': 'req-e5059cf2-41cc-49ce-bbf8-390d741a4608'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.682620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] The token cache is being invalidate because project ec72b0c2ad174f52b92b134c712ca543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:43.682910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.683583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e5059cf2-41cc-49ce-bbf8-390d741a4608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:43.685333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1205/2380] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:43 2022] DELETE /identity/v3/projects/ec72b0c2ad174f52b92b134c712ca543 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:43.691461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:43.691878 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:43.692244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:43.713273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:43.714475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:43.778194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1176/2381] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:43.787128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:43.787528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:43.787803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:43.808530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:43.809736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:43.822055 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1206/2382] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:43.824810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:43.837630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c6ed5-04b7-4bc5-91af-bf78344a4b09 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:43.842852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:43.843487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:43.843793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:43.844099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:43.844765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:43.845066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:43.846681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:43.865184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1177/2383] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:43.869754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:43.880400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157df08c-31a5-4e7f-b461-f9ec33ca3e46 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:43.885261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:43.885892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:43.886168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:43.886435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:43.887059 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:43.889209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:43.932141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1207/2384] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:43 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:43.933085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:43.945043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190af674-e2cc-46ff-ba76-bd5cf1623aa2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:43.959482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:43.961229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:43.980481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:44.037241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.038021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.038311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.038579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.039270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.041253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.042414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3752f0-a123-4410-8a40-9c940f811aef admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.047530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.048421 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.048864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.049311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] PATH_INFO: `/v3/users/791a3dc3f4b8427387d6c6c0b8fd549f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.051340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=791a3dc3f4b8427387d6c6c0b8fd549f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.054668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.075066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1178/2385] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:43 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:44.076042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.087643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78fbc17-0654-4c26-ad12-ad302328988e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.096672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.097521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.097850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.098246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.099022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.101104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.118614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1179/2386] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:44.119570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.131004 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a05da-6209-4c2e-bbcd-d9393047b900 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.131387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791a3dc3f4b8427387d6c6c0b8fd549f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.136069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.136682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.137137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.137568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.138165 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.140366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.148177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791a3dc3f4b8427387d6c6c0b8fd549f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.155162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1180/2387] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:44.159563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.173172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c99225-560f-4364-8c71-81f95a71475a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.177301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791a3dc3f4b8427387d6c6c0b8fd549f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.178359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791a3dc3f4b8427387d6c6c0b8fd549f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.179671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e08632-4ce0-4274-8742-9cc9750e4ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.179888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8881e089-b009-43b7-9e2c-9f3fe0383b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791a3dc3f4b8427387d6c6c0b8fd549f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.180505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08632-4ce0-4274-8742-9cc9750e4ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.180785 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08632-4ce0-4274-8742-9cc9750e4ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.181056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08632-4ce0-4274-8742-9cc9750e4ca9 admin admin] PATH_INFO: `/v3/projects/b54b312f07484aa69e679c4d6ebfe65e/users/39a2e1136f134225b8ae9bd829a0cf15/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.204086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1208/2388] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:43 2022] DELETE /identity/v3/users/791a3dc3f4b8427387d6c6c0b8fd549f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:44.207962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.209580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.211104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.227808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e08632-4ce0-4274-8742-9cc9750e4ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54b312f07484aa69e679c4d6ebfe65e, user_id=39a2e1136f134225b8ae9bd829a0cf15, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.231320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e08632-4ce0-4274-8742-9cc9750e4ca9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.233520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:44.259066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1181/2389] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:44 2022] PUT /identity/v3/projects/b54b312f07484aa69e679c4d6ebfe65e/users/39a2e1136f134225b8ae9bd829a0cf15/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:44.266797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:44.268130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.272919 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.273320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.274085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.329465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] MFA Rules not processed for user `39a2e1136f134225b8ae9bd829a0cf15`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:44.331012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.396546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1209/2390] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:44.407486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.408969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.483423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1182/2391] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:44.485125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.486585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.502109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec35d4d-8149-4bc4-ad41-917fbb55679a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.502983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195f7bc-51f4-4679-82e6-ee15fbcd4c6d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.505720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.506357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.506635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.506916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.508058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.508661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.509011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.509355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] PATH_INFO: `/v3/users/88d6e03c8d1d45f6827882925217f955` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.510834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d6e03c8d1d45f6827882925217f955)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.513339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.517363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.518812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.530039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1210/2392] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:44.531243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.532387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.568955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d6e03c8d1d45f6827882925217f955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.577933 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d6e03c8d1d45f6827882925217f955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.578848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d6e03c8d1d45f6827882925217f955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.581666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:44.601175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d6e03c8d1d45f6827882925217f955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.608897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645223-72ed-489b-8f8f-35bb6109a9a8 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.613193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-839ddc37-3023-4552-b7b3-6ce857e15e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d6e03c8d1d45f6827882925217f955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.615132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.615886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.616326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.616737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.618986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.621455 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1183/2393] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:44 2022] DELETE /identity/v3/users/88d6e03c8d1d45f6827882925217f955 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:44.626303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.626667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.627011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.635990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.637641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.646890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:44.648187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.648771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1211/2394] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:44.714815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1184/2395] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:44.726972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.729062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.747356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524f1ef-dfa3-4f7c-b06f-9437d0fd9931 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.810357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a151bb-f241-4fbe-867a-c5522ce1fa65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.811071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a151bb-f241-4fbe-867a-c5522ce1fa65 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.811484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a151bb-f241-4fbe-867a-c5522ce1fa65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.811849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a151bb-f241-4fbe-867a-c5522ce1fa65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.819166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:44.825548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a151bb-f241-4fbe-867a-c5522ce1fa65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.827125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a151bb-f241-4fbe-867a-c5522ce1fa65 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.841873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:44.843510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:44.843633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1212/2396] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:44.863848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3523d9-f674-45f5-9b0d-8db585dee03d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:44.878249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:44.879006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:44.879396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:44.879740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] PATH_INFO: `/v3/projects/4cdc754836b5483481ce53ad9d8843f8` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:44.881038 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdc754836b5483481ce53ad9d8843f8)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:44.883105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:44.944553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdc754836b5483481ce53ad9d8843f8', 'request_id': 'req-4b443b26-e798-459e-9f02-1a9fe20acc99'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.957310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdc754836b5483481ce53ad9d8843f8', 'request_id': 'req-4b443b26-e798-459e-9f02-1a9fe20acc99'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:44.979864 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdc754836b5483481ce53ad9d8843f8', 'request_id': 'req-4b443b26-e798-459e-9f02-1a9fe20acc99'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.005111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] The token cache is being invalidate because project 4cdc754836b5483481ce53ad9d8843f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:45.005437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.008349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4b443b26-e798-459e-9f02-1a9fe20acc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.010810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1185/2397] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:44 2022] DELETE /identity/v3/projects/4cdc754836b5483481ce53ad9d8843f8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:45.016358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.017100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.017100 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.070339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:45.071854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.149393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1213/2398] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:45.157541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.157902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.158250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.179368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:45.180636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.189365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1186/2399] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:45.193774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.204463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.206381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.220399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e45b4-4d7a-4faa-b59e-0bca72a9d71a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.223579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1bf48-10b4-4d3d-928d-3e441c3ea1af None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.226147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.227110 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.227460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.227770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] PATH_INFO: `/v3/projects/8cd6a73fa76c49df8bd90d7304876101` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.230290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd6a73fa76c49df8bd90d7304876101)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.231066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.231971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.232315 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.232739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.232937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.233707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:45.234077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.236529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.257026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1214/2400] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:00:45.262260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.284292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79bac0-28e7-427c-81db-f56b70b04303 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.293559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:45.297656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd6a73fa76c49df8bd90d7304876101', 'request_id': 'req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.311813 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd6a73fa76c49df8bd90d7304876101', 'request_id': 'req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.319073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd6a73fa76c49df8bd90d7304876101', 'request_id': 'req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.333139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] The token cache is being invalidate because project 8cd6a73fa76c49df8bd90d7304876101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:45.333586 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.334525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.338562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1187/2401] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:45 2022] DELETE /identity/v3/projects/8cd6a73fa76c49df8bd90d7304876101 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:45.342399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.345460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.368478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.369232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.369519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.370306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.371437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.373751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.419218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1215/2402] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:45 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:45.421283 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbfd05-90cf-488f-b5de-5ab53ea89fce admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.423442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.425437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.495073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.496014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.496293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.496563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] PATH_INFO: `/v3/users/2a4a3da2342d43ce96b007fb8f0896d7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.497746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4a3da2342d43ce96b007fb8f0896d7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.503059 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.508245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2581ac-7438-4ff5-818d-b40dabb856d6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.513448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.514126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.514412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.514689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.515349 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.517927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.529234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:45.543045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1216/2403] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:45 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:45.547338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.557188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b6d9d-924e-4b61-ab15-03fc1fefc52e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.559981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4a3da2342d43ce96b007fb8f0896d7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.562209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.563464 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.563747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.564017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.564658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.566519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.573048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4a3da2342d43ce96b007fb8f0896d7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.573921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4a3da2342d43ce96b007fb8f0896d7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.582181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1217/2404] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:45.583407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.591930 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4a3da2342d43ce96b007fb8f0896d7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.594872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38efe1-6a9f-4c94-ba45-72483a77ec8a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.599508 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.600106 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.600383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.600693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.601284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.602807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e9ec630-7b15-4351-a396-cef56e498019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4a3da2342d43ce96b007fb8f0896d7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:45.602935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.605906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:45.613670 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1218/2405] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:45.623518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1188/2406] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:45 2022] DELETE /identity/v3/users/2a4a3da2342d43ce96b007fb8f0896d7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:45.627637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.629278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.642561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310ca51-ab97-4374-a150-8df9bb5751d1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.700841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.701503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.701780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.702049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] PATH_INFO: `/v3/projects/ab33998afab84c9b9cd06db30e6e04a5/users/7a65371e0a864537897ef83292068010/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.717053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab33998afab84c9b9cd06db30e6e04a5, user_id=7a65371e0a864537897ef83292068010, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.720061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.733894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1219/2407] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:45 2022] PUT /identity/v3/projects/ab33998afab84c9b9cd06db30e6e04a5/users/7a65371e0a864537897ef83292068010/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:45.745766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.746052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.746327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.779926 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] MFA Rules not processed for user `7a65371e0a864537897ef83292068010`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:45.781318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.848632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1189/2408] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:45.859253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.861758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.897015 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:45.898508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:45.932190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843c83a-82f1-4dc5-a150-790af2d2611a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.936071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ac38e-bebb-41f1-b1b4-e7b77b99ce59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.936729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ac38e-bebb-41f1-b1b4-e7b77b99ce59 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.937005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ac38e-bebb-41f1-b1b4-e7b77b99ce59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.937272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ac38e-bebb-41f1-b1b4-e7b77b99ce59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.946605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ac38e-bebb-41f1-b1b4-e7b77b99ce59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.948111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ac38e-bebb-41f1-b1b4-e7b77b99ce59 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:45.958063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1220/2409] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:45.966854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ac8a4-6a55-4156-84f1-78673da5598e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:45.971451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:45.972073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:45.972357 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:45.972625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] PATH_INFO: `/v3/projects/2347287e49a140718a7eaa3e71feed43` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:45.973733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2347287e49a140718a7eaa3e71feed43)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:45.975696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.023914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2347287e49a140718a7eaa3e71feed43', 'request_id': 'req-d08f5ada-652e-48f2-a627-baf659a45e4b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:46.032517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2347287e49a140718a7eaa3e71feed43', 'request_id': 'req-d08f5ada-652e-48f2-a627-baf659a45e4b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:46.040738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2347287e49a140718a7eaa3e71feed43', 'request_id': 'req-d08f5ada-652e-48f2-a627-baf659a45e4b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:46.055065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] The token cache is being invalidate because project 2347287e49a140718a7eaa3e71feed43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:46.055488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:46.056068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d08f5ada-652e-48f2-a627-baf659a45e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:46.059267 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1190/2410] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:45 2022] DELETE /identity/v3/projects/2347287e49a140718a7eaa3e71feed43 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:46.064032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.064325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.064599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.100854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:46.102011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:46.155483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:46.173556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1221/2411] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:46.175009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.175317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.176213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.198254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:46.200227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:46.223787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1191/2412] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:46.228231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.238977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e029fff-e036-453d-9ac4-d10e7a15b41e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.243805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.244478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.244849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.245196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.245941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:46.246358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.248024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.261904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1222/2413] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:00:46.266241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.275640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7f4d5-779c-469a-adc9-a210a52e3008 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.280342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.280941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.281215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.282001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.282675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.284575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.297839 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:46.321759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1192/2414] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:46.325777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.338776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f256-97d3-4e50-8007-18edf252208f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.402139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.402764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.403027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.403348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.403975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.405896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.428200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1223/2415] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:46.432318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.442435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28fef-fd28-4e49-ab36-e288d3a4321c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.447614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.448205 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.448484 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.448754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.449387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.450969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.465088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1193/2416] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:46.466261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.478053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208878-dc33-41c8-9ce1-e877a51a621f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.482904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5093ffd-9a53-4bcf-9695-77f208a584be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.484154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5093ffd-9a53-4bcf-9695-77f208a584be admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.484154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5093ffd-9a53-4bcf-9695-77f208a584be admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.484359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5093ffd-9a53-4bcf-9695-77f208a584be admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.484850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5093ffd-9a53-4bcf-9695-77f208a584be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.486458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5093ffd-9a53-4bcf-9695-77f208a584be admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.497510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1224/2417] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:46.502178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.512625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30a67f-ec04-4ff6-9d02-edcf236de538 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.517910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.518565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.518848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.519122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] PATH_INFO: `/v3/projects/0942f999ba00479697041db0379eb1d0/users/5b064bcd63494dcf9c67ac3daca3d1e3/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.534597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0942f999ba00479697041db0379eb1d0, user_id=5b064bcd63494dcf9c67ac3daca3d1e3, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.537314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.553451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1194/2418] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:46 2022] PUT /identity/v3/projects/0942f999ba00479697041db0379eb1d0/users/5b064bcd63494dcf9c67ac3daca3d1e3/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:46.564164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.564682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.564802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.597837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] MFA Rules not processed for user `5b064bcd63494dcf9c67ac3daca3d1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:46.599451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:46.684697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1225/2419] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:46.689745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.810814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89846df7-a17f-4197-bf1a-a964cb3b76ef admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.818854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.819192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.819542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.820119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.821299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.823260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.853873 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1195/2420] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:46.857981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.870695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087cbd3-dad6-49ba-b98a-6bf8dc37568a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.926874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:46.939484 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.941260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.941557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.941812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.942893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.944903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:46.973948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1226/2421] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:46 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:46.975232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:46.990168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061135-72f7-4fa5-8372-a39462e80c74 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:46.995565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:46.996183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:46.996465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:46.996763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:46.997430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:46.999167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:47.011971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1196/2422] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:47.017914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:47.029720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae20dc-f067-49df-912a-9b97a7e80076 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:47.035367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:47.036045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.036323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.036592 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] PATH_INFO: `/v3/projects/d6c2d7b0a7d64739bd7fa30f2e9fc0f0/users/c1a6e50fa3664cc29877fbba2171060a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.055022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c2d7b0a7d64739bd7fa30f2e9fc0f0, user_id=c1a6e50fa3664cc29877fbba2171060a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:47.057780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:47.073126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1227/2423] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:47 2022] PUT /identity/v3/projects/d6c2d7b0a7d64739bd7fa30f2e9fc0f0/users/c1a6e50fa3664cc29877fbba2171060a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:47.077309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:47.088433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a0103-7d59-4756-ac81-0615edcae8c4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:47.146374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94775a-fba8-453e-8b31-e6d0aaea8c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:47.146989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94775a-fba8-453e-8b31-e6d0aaea8c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.147299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94775a-fba8-453e-8b31-e6d0aaea8c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.147578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94775a-fba8-453e-8b31-e6d0aaea8c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.148221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94775a-fba8-453e-8b31-e6d0aaea8c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:47.149881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94775a-fba8-453e-8b31-e6d0aaea8c92 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:47.159997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1197/2424] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:47.164064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:47.175835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62018ed-e8f0-4ae8-a6df-34dc32107e32 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:47.180748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:47.181400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.181679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.181949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] PATH_INFO: `/v3/projects/d6c2d7b0a7d64739bd7fa30f2e9fc0f0/users/c1a6e50fa3664cc29877fbba2171060a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.183871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c2d7b0a7d64739bd7fa30f2e9fc0f0, user_id=c1a6e50fa3664cc29877fbba2171060a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:47.186551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:47.198852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1228/2425] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:47 2022] PUT /identity/v3/projects/d6c2d7b0a7d64739bd7fa30f2e9fc0f0/users/c1a6e50fa3664cc29877fbba2171060a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:47.211354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.211718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.212009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.246791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] MFA Rules not processed for user `c1a6e50fa3664cc29877fbba2171060a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:47.248071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:47.265293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:47.337760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1198/2426] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:00:47.349589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:47.354833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:47.415741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1264-15dd-456b-a4ab-f624f96867f7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:47.419871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5b5cd-f6c4-4e9d-8191-cf500fd414b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:47.420538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b5cd-f6c4-4e9d-8191-cf500fd414b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.420752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b5cd-f6c4-4e9d-8191-cf500fd414b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.421026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b5cd-f6c4-4e9d-8191-cf500fd414b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.481407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b5cd-f6c4-4e9d-8191-cf500fd414b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:47.483890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b5cd-f6c4-4e9d-8191-cf500fd414b4 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:47.494623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1229/2427] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:57.331552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=22 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:57.489981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:10.121733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:10.196791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb09433-1046-40d6-a8d9-6458d213d718 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:10.200307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:10.202180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:10.202180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:10.202180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] PATH_INFO: `/v3/users/5b064bcd63494dcf9c67ac3daca3d1e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:10.204507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b064bcd63494dcf9c67ac3daca3d1e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:10.208327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:10.261467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b064bcd63494dcf9c67ac3daca3d1e3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.268836 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b064bcd63494dcf9c67ac3daca3d1e3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.269433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b064bcd63494dcf9c67ac3daca3d1e3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.282333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b064bcd63494dcf9c67ac3daca3d1e3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.289453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b064bcd63494dcf9c67ac3daca3d1e3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.295413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1199/2428] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:10 2022] DELETE /identity/v3/users/5b064bcd63494dcf9c67ac3daca3d1e3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:10.300782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:10.301267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:10.301370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:10.327506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:10.328757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:10.400720 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1230/2429] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:10.410893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:10.412382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:10.476101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0d45b-2d14-47d9-a1b5-9297ef632fca admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:10.480472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24263213-81b6-48a4-8f77-59123f02845e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:10.481223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24263213-81b6-48a4-8f77-59123f02845e service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:10.481521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24263213-81b6-48a4-8f77-59123f02845e service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:10.481874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24263213-81b6-48a4-8f77-59123f02845e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:10.491642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24263213-81b6-48a4-8f77-59123f02845e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:10.493169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24263213-81b6-48a4-8f77-59123f02845e service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:10.502622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1200/2430] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:01:10.802784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:10.804557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:10.822522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239db0c0-139a-4219-85a3-ed33d4c3d242 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:10.827536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:10.828821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:10.829109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:10.829461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] PATH_INFO: `/v3/projects/0942f999ba00479697041db0379eb1d0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:10.830617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0942f999ba00479697041db0379eb1d0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:10.832707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:10.955961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0942f999ba00479697041db0379eb1d0', 'request_id': 'req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.982438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0942f999ba00479697041db0379eb1d0', 'request_id': 'req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:10.985614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:10.986299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:10.986916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:10.989217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0942f999ba00479697041db0379eb1d0', 'request_id': 'req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.010364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] The token cache is being invalidate because project 0942f999ba00479697041db0379eb1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:11.016521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.017750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.020591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1231/2431] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:10 2022] DELETE /identity/v3/projects/0942f999ba00479697041db0379eb1d0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:11.024150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:11.025785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.041377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:11.043322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.064228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:11.129724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1201/2432] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:01:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:11.149119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5af6d1-c1e9-4a89-894f-38fce2c8b30c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:11.155448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:11.156139 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:11.156476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:11.157023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] PATH_INFO: `/v3/users/c1a6e50fa3664cc29877fbba2171060a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:11.158250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a6e50fa3664cc29877fbba2171060a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:11.160294 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:11.196553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:11.196726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:11.198278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.230328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a6e50fa3664cc29877fbba2171060a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.240449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a6e50fa3664cc29877fbba2171060a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.257331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a6e50fa3664cc29877fbba2171060a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.258382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a6e50fa3664cc29877fbba2171060a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.259245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a6e50fa3664cc29877fbba2171060a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.275870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1232/2433] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:11 2022] DELETE /identity/v3/users/c1a6e50fa3664cc29877fbba2171060a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:01:11.356386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f823c-5f29-4d0d-b48a-619887e07513 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:11.463738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:11.464419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:11.464729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:11.465072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:11.466957 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.550341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:11.552012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942378c4-54ab-4398-9846-4a5d21e2ba65 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:11.560373 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1202/2434] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:01:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:01:11.652755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:11.654436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.740611 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d88f70-1ddd-47f0-8f10-d34d0c050939 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:11.745666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:11.746302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:11.746582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:11.746860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] PATH_INFO: `/v3/projects/d6c2d7b0a7d64739bd7fa30f2e9fc0f0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:11.748109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c2d7b0a7d64739bd7fa30f2e9fc0f0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:11.750163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:11.809428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c2d7b0a7d64739bd7fa30f2e9fc0f0', 'request_id': 'req-9b2a3af2-02d9-4dbc-a143-d4b649d61239'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.819650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c2d7b0a7d64739bd7fa30f2e9fc0f0', 'request_id': 'req-9b2a3af2-02d9-4dbc-a143-d4b649d61239'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.826448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c2d7b0a7d64739bd7fa30f2e9fc0f0', 'request_id': 'req-9b2a3af2-02d9-4dbc-a143-d4b649d61239'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.840550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] The token cache is being invalidate because project d6c2d7b0a7d64739bd7fa30f2e9fc0f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:11.840868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.841509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9b2a3af2-02d9-4dbc-a143-d4b649d61239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:11.846425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1233/2435] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:11 2022] DELETE /identity/v3/projects/d6c2d7b0a7d64739bd7fa30f2e9fc0f0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:11.853245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:11.853529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:11.854060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:11.877128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:11.878433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.887437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:11.951270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1203/2436] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:11.960553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:11.961130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:11.961446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:11.987846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:11.989337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:11.998418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1234/2437] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:12.004124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.015294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ece39-ea6d-42e2-ac43-d35e5c3f41ad None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.036607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.037316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.038261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.039397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.040132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:01:12.040648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.044167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.058394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1204/2438] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:01:12.062701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.073495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae295d0-d388-4149-b716-61ab6777be8e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.080050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd91fe8-8159-4a30-9633-519e0941ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.080813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd91fe8-8159-4a30-9633-519e0941ee9d admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.081109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd91fe8-8159-4a30-9633-519e0941ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.081388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd91fe8-8159-4a30-9633-519e0941ee9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.082034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd91fe8-8159-4a30-9633-519e0941ee9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.083998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd91fe8-8159-4a30-9633-519e0941ee9d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.127097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1235/2439] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:01:12 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:01:12.131492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.143625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c8795-bdda-408b-a1cc-a8043d1dd0f1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.205098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.205542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.206126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.208496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.209177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.209657 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.209895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.210377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.212386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.215997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:12.230781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:12.232012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.235256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1205/2440] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:01:12 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:01:12.239144 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.249415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a517d7-5cdb-46ab-99bc-1e461b325448 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.254101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.254923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.254923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.255223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.256042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.257893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.269421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1206/2441] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:01:12.273322 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.283225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191b248-5869-41ab-be15-9e2a42b69876 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.287747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.288309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.288585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.288853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.289469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.291016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.291437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1236/2442] 10.222.0.66 () {58 vars in 967 bytes} [Thu Sep 22 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:12.303242 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1207/2443] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:01:12.305239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.305638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.307291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.317352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f0b80-c03b-4453-9cc5-ea2120ec4045 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.324274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac28ab-4841-4435-a5cb-a0d49078d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.324942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac28ab-4841-4435-a5cb-a0d49078d1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.325225 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac28ab-4841-4435-a5cb-a0d49078d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.325499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac28ab-4841-4435-a5cb-a0d49078d1f3 admin admin] PATH_INFO: `/v3/projects/91a6cea77d0f436390ceb0952327fbaa/users/48d227ba56af4907aa205833b50ed315/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.345098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac28ab-4841-4435-a5cb-a0d49078d1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a6cea77d0f436390ceb0952327fbaa, user_id=48d227ba56af4907aa205833b50ed315, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.348653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac28ab-4841-4435-a5cb-a0d49078d1f3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.367539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1237/2444] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:01:12 2022] PUT /identity/v3/projects/91a6cea77d0f436390ceb0952327fbaa/users/48d227ba56af4907aa205833b50ed315/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:12.372827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.373326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.373802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.397140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe6b37-ab94-4d30-815a-2a5ed6fb6760 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.397815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:12.399785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.404628 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:12.467641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1238/2445] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:12.476398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9ecb7-1544-4707-a126-d3a028630a99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.477376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9ecb7-1544-4707-a126-d3a028630a99 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.477799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9ecb7-1544-4707-a126-d3a028630a99 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.478085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9ecb7-1544-4707-a126-d3a028630a99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.478498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.480389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.545195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473b79e-e9a2-4e5e-8bc8-2ef1d161b661 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.549687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.550310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.550586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.550853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.555981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9ecb7-1544-4707-a126-d3a028630a99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.557849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9ecb7-1544-4707-a126-d3a028630a99 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.563712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.565136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.567016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1208/2446] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 15:01:12.575415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1239/2447] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:12.894902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cf27c-e61a-4a92-a4f0-e5747c7a2ce3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.895194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cf27c-e61a-4a92-a4f0-e5747c7a2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.895511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cf27c-e61a-4a92-a4f0-e5747c7a2ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.919987 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d06cf27c-e61a-4a92-a4f0-e5747c7a2ce3 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:12.921910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d06cf27c-e61a-4a92-a4f0-e5747c7a2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.934673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1209/2448] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:12.950398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.979169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05eba-386f-487f-ae58-ec0d96214411 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.990504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ca069-b7f0-407c-8ac3-4da5220f9359 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.991504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ca069-b7f0-407c-8ac3-4da5220f9359 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.992163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ca069-b7f0-407c-8ac3-4da5220f9359 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.993306 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ca069-b7f0-407c-8ac3-4da5220f9359 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:13.006665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ca069-b7f0-407c-8ac3-4da5220f9359 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:13.008820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ca069-b7f0-407c-8ac3-4da5220f9359 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:13.019068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1240/2449] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:01:13.333654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:13.334119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:13.334220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:13.370664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] MFA Rules not processed for user `48d227ba56af4907aa205833b50ed315`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:13.371909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:13.419038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:13.419347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:13.419687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:13.443984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:13.446493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:13.454684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1241/2450] 10.222.0.66 () {58 vars in 967 bytes} [Thu Sep 22 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:13.455176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1210/2451] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:13.467266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:13.468069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:13.474414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:13.488343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333da4b-c5de-4c11-a937-6476a5aa6222 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:13.493650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae7baf-b563-4b38-920f-781069676226 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:13.494266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae7baf-b563-4b38-920f-781069676226 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:13.494549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae7baf-b563-4b38-920f-781069676226 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:13.494830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae7baf-b563-4b38-920f-781069676226 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:13.502713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:13.506460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae7baf-b563-4b38-920f-781069676226 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:13.511984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae7baf-b563-4b38-920f-781069676226 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:13.520903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:13.528496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1242/2452] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:01:13.552089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d2b68-5ac0-4131-b73f-b00235018fe1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:13.563464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:13.564270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:13.564640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:13.571615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:13.582112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:13.583558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:13.593227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1211/2453] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:01:15.196810 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4d0c-4f3b-4777-af41-4e1f72491723 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:15.197280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4d0c-4f3b-4777-af41-4e1f72491723 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:15.197391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4d0c-4f3b-4777-af41-4e1f72491723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:15.217274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a12c4d0c-4f3b-4777-af41-4e1f72491723 None None] MFA Rules not processed for user `dab1b8b77cfb479098ac9e0d785a25c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:15.218535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a12c4d0c-4f3b-4777-af41-4e1f72491723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:15.273438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1243/2454] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:23.589126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=11 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:24.191832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:24.193866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:24.208288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b1218-da28-43ca-9176-d755e87adfee service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:24.214039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:24.214846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:24.215227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:24.215571 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] PATH_INFO: `/v3/users/7a65371e0a864537897ef83292068010` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:24.216825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a65371e0a864537897ef83292068010)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:24.218937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:24.288811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a65371e0a864537897ef83292068010'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:24.303393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a65371e0a864537897ef83292068010'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:24.304158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a65371e0a864537897ef83292068010'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:24.327061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a65371e0a864537897ef83292068010'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:24.338303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a65371e0a864537897ef83292068010'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:24.345647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1212/2455] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:24 2022] DELETE /identity/v3/users/7a65371e0a864537897ef83292068010 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:01:24.350858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3cc8-20e0-431d-bc2a-93e5e5896d2d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:24.351752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3cc8-20e0-431d-bc2a-93e5e5896d2d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:24.352216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3cc8-20e0-431d-bc2a-93e5e5896d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:24.376127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-99ab3cc8-20e0-431d-bc2a-93e5e5896d2d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:24.377332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-99ab3cc8-20e0-431d-bc2a-93e5e5896d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:24.453367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1244/2456] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:24.464661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:24.466390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:24.550760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26abe7-bd87-45fc-b11b-6d8b581e3020 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:24.557249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:24.558226 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:24.558607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:24.559303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:24.562905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:24.563227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:24.563536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:24.572265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:24.574066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:24.589157 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1213/2457] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:24.589752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] MFA Rules not processed for user `48d227ba56af4907aa205833b50ed315`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:24.590749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:24.695118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1245/2458] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:24.941680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:24.945400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:24.963450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5939fb-35d6-40c8-b7e6-2961e0a45ffe service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:24.970725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:24.972598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:24.972893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:24.973169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] PATH_INFO: `/v3/projects/ab33998afab84c9b9cd06db30e6e04a5` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:24.974362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab33998afab84c9b9cd06db30e6e04a5)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:24.983937 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:25.052298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab33998afab84c9b9cd06db30e6e04a5', 'request_id': 'req-47842d99-dead-4695-8080-48a7bc86c589'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.069036 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab33998afab84c9b9cd06db30e6e04a5', 'request_id': 'req-47842d99-dead-4695-8080-48a7bc86c589'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.078309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab33998afab84c9b9cd06db30e6e04a5', 'request_id': 'req-47842d99-dead-4695-8080-48a7bc86c589'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.087545 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:25.097461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] The token cache is being invalidate because project ab33998afab84c9b9cd06db30e6e04a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:25.097886 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.099729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-47842d99-dead-4695-8080-48a7bc86c589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.103264 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1214/2459] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:24 2022] DELETE /identity/v3/projects/ab33998afab84c9b9cd06db30e6e04a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:01:25.365953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:25.367654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:25.382237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3be2e4-429d-4396-be4c-d2c0fc5c432f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:25.488686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:25.503528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:25.504275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:25.504638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:25.505001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] PATH_INFO: `/v3/users/6aa62784a6fa45b4b37b8af70ab16e31` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:25.507270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa62784a6fa45b4b37b8af70ab16e31)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:25.511485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:25.562560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa62784a6fa45b4b37b8af70ab16e31'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.570958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa62784a6fa45b4b37b8af70ab16e31'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.585842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa62784a6fa45b4b37b8af70ab16e31'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.586645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa62784a6fa45b4b37b8af70ab16e31'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.587744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa62784a6fa45b4b37b8af70ab16e31'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:25.602582 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1246/2460] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:25 2022] DELETE /identity/v3/users/6aa62784a6fa45b4b37b8af70ab16e31 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:25.607334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:25.608147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:25.608603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:25.657411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:25.658970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:25.738504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1215/2461] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:25.749471 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:25.751188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:25.836453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac731282-a753-4cc0-8ddc-51358cbe574d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:25.842965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e323e-3d0e-4c4a-8c2b-417079f27ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:25.842965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e323e-3d0e-4c4a-8c2b-417079f27ecb service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:25.843650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e323e-3d0e-4c4a-8c2b-417079f27ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:25.844028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e323e-3d0e-4c4a-8c2b-417079f27ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:25.855082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e323e-3d0e-4c4a-8c2b-417079f27ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:25.856763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e323e-3d0e-4c4a-8c2b-417079f27ecb service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:25.870260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1247/2462] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:26.071240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.072568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:26.086971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a99e4-8d4d-4deb-a55e-c9878a082baf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.093461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.095077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.095077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.095077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] PATH_INFO: `/v3/projects/a7dc539727694ed1b3d70900860673bd` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.096576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7dc539727694ed1b3d70900860673bd)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.098923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.152080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7dc539727694ed1b3d70900860673bd', 'request_id': 'req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:26.160608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7dc539727694ed1b3d70900860673bd', 'request_id': 'req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:26.167661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7dc539727694ed1b3d70900860673bd', 'request_id': 'req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:26.180745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] The token cache is being invalidate because project a7dc539727694ed1b3d70900860673bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:26.181039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:26.181776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-15e8963a-4a31-49a2-a3e8-fd2fdb2fdce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:26.183718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1216/2463] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:26 2022] DELETE /identity/v3/projects/a7dc539727694ed1b3d70900860673bd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:26.188929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.189220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.189500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.213918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:26.215429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:26.281989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1248/2464] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:26.290278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.290568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.290850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.311739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:26.313107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:26.322131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1217/2465] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:26.326988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.337323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84d1f-e579-4c04-9e3e-e43da1d2ca4a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.342374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.343035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.343353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.343623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.344331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:01:26.344682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.346467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.350107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:26.364170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1249/2466] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:01:26.368759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.379477 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be0c19-7567-4d58-8c43-fe49fe77d176 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.384439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.385332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.385611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.385876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.386532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.388483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.420665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1218/2467] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:01:26 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:01:26.422483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.433387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1765bc-e4bc-4fb9-96a0-802657cc3a29 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.516821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.517971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.518094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.519043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.519851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.522165 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.551403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1250/2468] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:01:26 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:01:26.556542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.569371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb949df0-73ed-4023-af71-1b5b2a3fa484 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.574771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.575605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.576035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.576625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.578120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.582115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.595176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1219/2469] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:01:26.599346 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.611025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79264b8-8624-4829-a512-db1dda040cbf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.617332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d13c2f-3a39-4d30-b615-af99114c9bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.618104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13c2f-3a39-4d30-b615-af99114c9bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.618698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13c2f-3a39-4d30-b615-af99114c9bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.619169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d13c2f-3a39-4d30-b615-af99114c9bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.619948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d13c2f-3a39-4d30-b615-af99114c9bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.621813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d13c2f-3a39-4d30-b615-af99114c9bbb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.638235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1251/2470] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:01:26.641339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.656616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f539e6-3b53-4495-a978-9c3946d24c29 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.662074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.662811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.663103 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.663414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] PATH_INFO: `/v3/projects/8abb837ab9f041f18001242b3ba112c0/users/cc3e1c8beeef492282f98407c22fb682/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.684074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abb837ab9f041f18001242b3ba112c0, user_id=cc3e1c8beeef492282f98407c22fb682, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.687918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.714083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1220/2471] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:01:26 2022] PUT /identity/v3/projects/8abb837ab9f041f18001242b3ba112c0/users/cc3e1c8beeef492282f98407c22fb682/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:01:26.728876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.729207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.729543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.758515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] MFA Rules not processed for user `cc3e1c8beeef492282f98407c22fb682`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:26.760119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:26.825543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1252/2472] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:26.837064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:26.839009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:26.860145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:26.901122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27640e9b-ee31-4f79-b886-cb783d32d83d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:26.905057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50470379-6ce9-4e83-9e68-d8ae2b25d4ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:26.905673 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50470379-6ce9-4e83-9e68-d8ae2b25d4ef service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:26.905958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50470379-6ce9-4e83-9e68-d8ae2b25d4ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:26.906224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50470379-6ce9-4e83-9e68-d8ae2b25d4ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:26.917166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50470379-6ce9-4e83-9e68-d8ae2b25d4ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:26.918787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50470379-6ce9-4e83-9e68-d8ae2b25d4ef service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:26.928885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1221/2473] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:27.848438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:27.850607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:27.921574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67355567-40c0-42df-b274-e25938e0c6f1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:27.927010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:27.927523 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:27.927859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:27.928140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] PATH_INFO: `/v3/users/39a2e1136f134225b8ae9bd829a0cf15` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:27.929278 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a2e1136f134225b8ae9bd829a0cf15)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:27.931285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:27.936107 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:27.978781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a2e1136f134225b8ae9bd829a0cf15'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:27.985398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a2e1136f134225b8ae9bd829a0cf15'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:27.998509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a2e1136f134225b8ae9bd829a0cf15'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:27.999316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a2e1136f134225b8ae9bd829a0cf15'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:27.999863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a2e1136f134225b8ae9bd829a0cf15'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:28.012364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1253/2474] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:27 2022] DELETE /identity/v3/users/39a2e1136f134225b8ae9bd829a0cf15 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:28.017125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:28.017616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:28.017724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:28.037483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:28.038679 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:28.100046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1222/2475] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:28.110004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:28.113335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:28.173496 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc089a-8525-4cef-8aab-660a71f93c3f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:28.177826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:28.178431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:28.178710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:28.178984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:28.187903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:28.189361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:28.199248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1254/2476] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:28.489033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:28.491045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:28.524424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82396b-3054-4960-acbe-4f69b6e70afe None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:28.529222 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:28.529875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:28.530184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:28.530693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] PATH_INFO: `/v3/projects/b54b312f07484aa69e679c4d6ebfe65e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:28.532336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54b312f07484aa69e679c4d6ebfe65e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:28.535435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:28.588803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54b312f07484aa69e679c4d6ebfe65e', 'request_id': 'req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:28.595715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54b312f07484aa69e679c4d6ebfe65e', 'request_id': 'req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:28.602118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54b312f07484aa69e679c4d6ebfe65e', 'request_id': 'req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:28.620307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] The token cache is being invalidate because project b54b312f07484aa69e679c4d6ebfe65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:28.620700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:28.621320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:28.623358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1223/2477] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:28 2022] DELETE /identity/v3/projects/b54b312f07484aa69e679c4d6ebfe65e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:28.760807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:28.763149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:28.861313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e423f-7204-4171-8c8e-797a169f0e82 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:28.898430 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:28.948927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:28.949555 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:28.950291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:28.950291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:28.951931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:28.969180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:28.973558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04af63-b0cf-4711-9101-db3395c87a70 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:28.986789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1255/2478] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:29.170220 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:29.171884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:29.228675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:29.248276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ac96b-8ad1-4eda-bdca-776a7d5ecaa2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:29.253221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:29.253841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:29.254117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:29.254387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] PATH_INFO: `/v3/users/9959f9018f224e1983a2a3775f2b79a3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:29.255520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9959f9018f224e1983a2a3775f2b79a3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:29.257522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:29.305597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9959f9018f224e1983a2a3775f2b79a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.312856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9959f9018f224e1983a2a3775f2b79a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.313530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9959f9018f224e1983a2a3775f2b79a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.325699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9959f9018f224e1983a2a3775f2b79a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.333057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9959f9018f224e1983a2a3775f2b79a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.338902 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1224/2479] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:29 2022] DELETE /identity/v3/users/9959f9018f224e1983a2a3775f2b79a3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:29.343386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:29.343677 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:29.343951 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:29.363634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:29.364975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:29.424502 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1256/2480] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:29.434053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:29.435472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:29.515057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904f37a-3975-4d76-99af-5d3b69984684 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:29.519265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf92c23-7e9a-4c18-92f7-04a02fb51733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:29.519949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf92c23-7e9a-4c18-92f7-04a02fb51733 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:29.520324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf92c23-7e9a-4c18-92f7-04a02fb51733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:29.520655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf92c23-7e9a-4c18-92f7-04a02fb51733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:29.530442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf92c23-7e9a-4c18-92f7-04a02fb51733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:29.532379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf92c23-7e9a-4c18-92f7-04a02fb51733 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:29.543301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1225/2481] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:29.812548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:29.814180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:29.826914 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5d5c0-0abd-4422-847d-629e9776a132 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:29.832916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:29.833545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:29.833820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:29.834092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] PATH_INFO: `/v3/projects/a5908d1ed3d54f38a312fe4ee0208e76` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:29.835258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5908d1ed3d54f38a312fe4ee0208e76)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:29.837386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:29.886780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5908d1ed3d54f38a312fe4ee0208e76', 'request_id': 'req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.894250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5908d1ed3d54f38a312fe4ee0208e76', 'request_id': 'req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.900063 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5908d1ed3d54f38a312fe4ee0208e76', 'request_id': 'req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.910901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] The token cache is being invalidate because project a5908d1ed3d54f38a312fe4ee0208e76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:29.911188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.911836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-de568ad5-14f5-4654-af31-5aa9ec2fd7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:29.913313 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1257/2482] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:29 2022] DELETE /identity/v3/projects/a5908d1ed3d54f38a312fe4ee0208e76 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:29.918119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:29.918414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:29.918691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:29.939092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:29.940788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:29.998505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1226/2483] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:30.006832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.007245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.007536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.027064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:30.028316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:30.036247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1258/2484] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:30.040580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.051265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a31a97-7164-41c0-9549-ddade9084bca None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.056130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.056744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.057020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.057295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.057944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:01:30.058245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.060168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.061812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:30.075807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1227/2485] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:01:30.080105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.089251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cceed48-ee2f-424c-8eb1-e3a7a70f5f47 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.094032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.094632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.094911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.095183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.095847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.097818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.125374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1259/2486] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:01:30 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:01:30.130179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.140196 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c3195-3b2d-4294-9ea8-4d0f5c364ba0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.198691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.199367 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.199614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.199908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.200544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.202486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.226156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1228/2487] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:01:30 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:01:30.230297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.241994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcac01-4424-479a-9ba0-2137ebea8333 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.250957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.251693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.252026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.252296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.252994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.254669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.268439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1260/2488] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:01:30.269618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.279534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cdc9b-4faa-401f-85c5-3882b1abd860 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.284714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bff43-3729-4f43-8626-1d538f3e9bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.285312 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bff43-3729-4f43-8626-1d538f3e9bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.285594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bff43-3729-4f43-8626-1d538f3e9bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.285870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bff43-3729-4f43-8626-1d538f3e9bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.286508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bff43-3729-4f43-8626-1d538f3e9bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.288146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bff43-3729-4f43-8626-1d538f3e9bf2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.298996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1229/2489] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:01:30.303989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.309220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:30.316772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395fc0f-557c-408e-9283-f33cfb40b95e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.323192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.323881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.324154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.324425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] PATH_INFO: `/v3/projects/5d81a4e464e64ae9b5d9068936bb8965/users/5155817e56924ea08149ab31766523bd/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.339194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d81a4e464e64ae9b5d9068936bb8965, user_id=5155817e56924ea08149ab31766523bd, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.341857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.356028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1261/2490] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:01:30 2022] PUT /identity/v3/projects/5d81a4e464e64ae9b5d9068936bb8965/users/5155817e56924ea08149ab31766523bd/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:30.368260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544309f-d632-4837-a205-befc7a41c82f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.368550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544309f-d632-4837-a205-befc7a41c82f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.368826 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544309f-d632-4837-a205-befc7a41c82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.400459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0544309f-d632-4837-a205-befc7a41c82f None None] MFA Rules not processed for user `5155817e56924ea08149ab31766523bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:30.401694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0544309f-d632-4837-a205-befc7a41c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:30.473187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1230/2491] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:30.484432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.486110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:30.542190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea9171-e10c-4e38-8205-517eb7068425 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.545834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771fddeb-b436-4054-90f3-eae8193bcfc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.546435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fddeb-b436-4054-90f3-eae8193bcfc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.546717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fddeb-b436-4054-90f3-eae8193bcfc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.546990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771fddeb-b436-4054-90f3-eae8193bcfc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.555290 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fddeb-b436-4054-90f3-eae8193bcfc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.556736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771fddeb-b436-4054-90f3-eae8193bcfc1 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.565881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1262/2492] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:36.183796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544309f-d632-4837-a205-befc7a41c82f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:36.185202 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0544309f-d632-4837-a205-befc7a41c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:36.250965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544309f-d632-4837-a205-befc7a41c82f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:36.255942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:36.256558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:36.256831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:36.257099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] PATH_INFO: `/v3/users/cc3e1c8beeef492282f98407c22fb682` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:36.258194 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc3e1c8beeef492282f98407c22fb682)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:36.260421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:36.262614 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:36.306411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc3e1c8beeef492282f98407c22fb682'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.313333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc3e1c8beeef492282f98407c22fb682'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.313967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc3e1c8beeef492282f98407c22fb682'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.326258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc3e1c8beeef492282f98407c22fb682'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.333651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc3e1c8beeef492282f98407c22fb682'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.340497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1231/2493] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:36 2022] DELETE /identity/v3/users/cc3e1c8beeef492282f98407c22fb682 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:36.344893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:36.345407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:36.345511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:36.365848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:36.367122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:36.428824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1263/2494] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:36.438422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:36.440451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:36.503526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2275659-c325-401e-97c3-97b992d83237 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:36.507381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd8a69-ab48-4785-a99e-8582d32088ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:36.508024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8a69-ab48-4785-a99e-8582d32088ac service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:36.508327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8a69-ab48-4785-a99e-8582d32088ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:36.508596 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8a69-ab48-4785-a99e-8582d32088ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:36.517778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd8a69-ab48-4785-a99e-8582d32088ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:36.520290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd8a69-ab48-4785-a99e-8582d32088ac service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:36.529625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1232/2495] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:01:36.835044 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:36.836526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:36.849120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b8d3-cd22-4351-8857-84ab3a77e700 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:36.853889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:36.854579 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:36.854879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:36.855311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] PATH_INFO: `/v3/projects/8abb837ab9f041f18001242b3ba112c0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:36.856885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8abb837ab9f041f18001242b3ba112c0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:36.858632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:36.920254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abb837ab9f041f18001242b3ba112c0', 'request_id': 'req-a5320076-d572-43fc-b50f-10e95d7012b0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.929374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abb837ab9f041f18001242b3ba112c0', 'request_id': 'req-a5320076-d572-43fc-b50f-10e95d7012b0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.937583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abb837ab9f041f18001242b3ba112c0', 'request_id': 'req-a5320076-d572-43fc-b50f-10e95d7012b0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.949849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] The token cache is being invalidate because project 8abb837ab9f041f18001242b3ba112c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:36.950138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.950768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5320076-d572-43fc-b50f-10e95d7012b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:36.952286 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1264/2496] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:36 2022] DELETE /identity/v3/projects/8abb837ab9f041f18001242b3ba112c0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:46.525502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:46.946868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:54.766221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:54.766723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:54.766890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:54.786300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] MFA Rules not processed for user `48d227ba56af4907aa205833b50ed315`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:54.787525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:54.840655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1233/2497] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:04.833755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=25 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:19.656215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:19.656707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:19.656812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:19.677857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:19.680468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:19.754831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1265/2498] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:19.814932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:19.816352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:19.902501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6592cb4-d748-4a14-9e77-c6320aed2408 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:19.902501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4beb4f6-84f6-4a65-8fba-90a3908de8db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:19.902501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beb4f6-84f6-4a65-8fba-90a3908de8db service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:19.902501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beb4f6-84f6-4a65-8fba-90a3908de8db service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:19.902872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beb4f6-84f6-4a65-8fba-90a3908de8db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:19.910315 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4beb4f6-84f6-4a65-8fba-90a3908de8db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:19.911936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4beb4f6-84f6-4a65-8fba-90a3908de8db service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:19.922425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1234/2499] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:02:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:02:25.064756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:25.067673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:25.153171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625d03-2170-44e5-b0fa-e9f6c5985c7e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:25.158516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:25.159239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:25.159662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:25.159939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] PATH_INFO: `/v3/users/5155817e56924ea08149ab31766523bd` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:25.161155 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5155817e56924ea08149ab31766523bd)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:25.163111 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:25.220816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5155817e56924ea08149ab31766523bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:25.231749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5155817e56924ea08149ab31766523bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:25.248103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5155817e56924ea08149ab31766523bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:25.248836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5155817e56924ea08149ab31766523bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:25.249636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5155817e56924ea08149ab31766523bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:25.264589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1266/2500] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:02:25 2022] DELETE /identity/v3/users/5155817e56924ea08149ab31766523bd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:25.268906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:25.269474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:25.269474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:25.289583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:25.290812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:25.357721 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1235/2501] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:25.367558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:25.369102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:25.402316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:25.435060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a4c4c-37e5-44d0-ae94-f88d142516b8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:25.438724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f32c04-3c3c-4577-a17c-58a141cd3667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:25.439379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f32c04-3c3c-4577-a17c-58a141cd3667 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:25.439665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f32c04-3c3c-4577-a17c-58a141cd3667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:25.439927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f32c04-3c3c-4577-a17c-58a141cd3667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:25.450197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f32c04-3c3c-4577-a17c-58a141cd3667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:25.451688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f32c04-3c3c-4577-a17c-58a141cd3667 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:25.462061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1267/2502] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:02:25.876443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:25.878151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:25.907787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f833d46-fbf0-43d9-b3de-a55b9452ff17 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:25.919790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:25.920559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:25.920837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:25.921115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] PATH_INFO: `/v3/projects/5d81a4e464e64ae9b5d9068936bb8965` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:25.922375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d81a4e464e64ae9b5d9068936bb8965)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:25.924463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:25.984634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d81a4e464e64ae9b5d9068936bb8965', 'request_id': 'req-3f09a952-912f-4029-a48e-8988970df862'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:25.993417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d81a4e464e64ae9b5d9068936bb8965', 'request_id': 'req-3f09a952-912f-4029-a48e-8988970df862'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:26.000780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d81a4e464e64ae9b5d9068936bb8965', 'request_id': 'req-3f09a952-912f-4029-a48e-8988970df862'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:26.012833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] The token cache is being invalidate because project 5d81a4e464e64ae9b5d9068936bb8965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:02:26.013122 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:26.013795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3f09a952-912f-4029-a48e-8988970df862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:26.015733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1236/2503] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:02:25 2022] DELETE /identity/v3/projects/5d81a4e464e64ae9b5d9068936bb8965 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:26.021676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.022209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.022599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.045735 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:26.046408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:26.135690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1268/2504] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:26.136769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.137055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.137329 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.162929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:26.162929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:26.176876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1237/2505] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:02:26.178431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.190530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee44fb-3ff9-499a-820b-d58ecc5c8bcc None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.195801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.196415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.196699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.196973 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.197854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:02:26.198172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.199827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.218684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1269/2506] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:02:26.226644 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.236989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72197a17-268b-4df8-a6b0-7d8c4f85e063 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.242562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.245395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.245395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.246041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.247818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.250509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.258643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:26.295872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1238/2507] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:02:26 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:26.297946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.309230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01eb6ff-63cc-423d-81d6-756cc4f336c6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.377884 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.378565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.378767 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.379016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.379843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.381763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.411238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1270/2508] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:02:26 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:26.414621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.425994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7e89-23d4-4c19-bc26-aa3736a70fdc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.431460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.432078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.432453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.432726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.433369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.435020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.451656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1239/2509] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:02:26.453157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.465770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ded3a3-02cc-48c0-b9bc-cdc58d396f56 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.472455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc416b-98ca-4686-8ad3-a8e10ee9a15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.472455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc416b-98ca-4686-8ad3-a8e10ee9a15a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.472455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc416b-98ca-4686-8ad3-a8e10ee9a15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.472455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc416b-98ca-4686-8ad3-a8e10ee9a15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.472904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc416b-98ca-4686-8ad3-a8e10ee9a15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.476809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc416b-98ca-4686-8ad3-a8e10ee9a15a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.495252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1271/2510] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:02:26.497087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.512011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636f3d6-8f02-4cbc-9ff8-2295308355ef admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.520168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.520843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.521124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.521395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] PATH_INFO: `/v3/projects/607ab875b1054309b32f91e7ca2515e9/users/d0f67a1377e84c4cabc0ee56f6313980/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.538265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=607ab875b1054309b32f91e7ca2515e9, user_id=d0f67a1377e84c4cabc0ee56f6313980, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.541580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.556173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1240/2511] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:02:26 2022] PUT /identity/v3/projects/607ab875b1054309b32f91e7ca2515e9/users/d0f67a1377e84c4cabc0ee56f6313980/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:02:26.567587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f84913-8285-4318-b154-e45be9941b89 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.568024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f84913-8285-4318-b154-e45be9941b89 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.568422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f84913-8285-4318-b154-e45be9941b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.607686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f4f84913-8285-4318-b154-e45be9941b89 None None] MFA Rules not processed for user `d0f67a1377e84c4cabc0ee56f6313980`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:26.609015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f4f84913-8285-4318-b154-e45be9941b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:26.638690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:26.707364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1272/2512] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:26.718606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:26.720383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:26.792305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd77ad-37d3-4a6c-8408-426289eee5a5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:26.796301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:26.797111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:26.797530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:26.797807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:26.809565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:26.811866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:26.824097 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1241/2513] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:02:36.698956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=17 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:36.819920 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:39.309455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e5424-1c44-4253-b8ba-7212a06c6b1f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:39.310127 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e5424-1c44-4253-b8ba-7212a06c6b1f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:39.310719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e5424-1c44-4253-b8ba-7212a06c6b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:39.331682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6e8e5424-1c44-4253-b8ba-7212a06c6b1f None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:39.332913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6e8e5424-1c44-4253-b8ba-7212a06c6b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:39.388446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1273/2514] 10.222.0.66 () {58 vars in 967 bytes} [Thu Sep 22 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:39.401641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:39.403047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:39.470590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb41b4f-33dd-4b4d-b158-ae9657b08453 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:39.475944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde1328-0271-42dc-92e8-33a301ab7787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:39.476553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde1328-0271-42dc-92e8-33a301ab7787 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:39.476842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde1328-0271-42dc-92e8-33a301ab7787 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:39.477155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde1328-0271-42dc-92e8-33a301ab7787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:39.485435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde1328-0271-42dc-92e8-33a301ab7787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:39.487006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde1328-0271-42dc-92e8-33a301ab7787 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:39.494888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1242/2515] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:02:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:02:46.721826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fefb5-da38-4da8-bfff-8470a823a38e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:46.722325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fefb5-da38-4da8-bfff-8470a823a38e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:46.722455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fefb5-da38-4da8-bfff-8470a823a38e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:46.723786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1274/2516] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 15:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:02:46.752496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:46.752981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:46.753083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:46.773411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] MFA Rules not processed for user `a3f43e7a1e9840299697271428a67872`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:46.774625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:46.786553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fefb5-da38-4da8-bfff-8470a823a38e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:46.790780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c12fefb5-da38-4da8-bfff-8470a823a38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:46.855074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1243/2517] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:02:46.867340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:46.869204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:46.884022 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e4d5-b7f4-427f-9cec-1276207b29ec None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:46.885858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fefb5-da38-4da8-bfff-8470a823a38e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:46.890407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:46.890972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:46.891098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:46.891345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:46.891624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] PATH_INFO: `/v3/users/48d227ba56af4907aa205833b50ed315` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:46.891732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:46.891997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:46.892272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:46.892793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d227ba56af4907aa205833b50ed315)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:46.894728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:46.903236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:46.904845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:46.915431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1244/2518] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:02:46.948039 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d227ba56af4907aa205833b50ed315'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:46.957291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d227ba56af4907aa205833b50ed315'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:46.970089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d227ba56af4907aa205833b50ed315'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:46.970762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d227ba56af4907aa205833b50ed315'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:46.971409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-77db052b-5a6d-4db4-b49d-4981631d3404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d227ba56af4907aa205833b50ed315'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:46.983885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1275/2519] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:02:46 2022] DELETE /identity/v3/users/48d227ba56af4907aa205833b50ed315 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:47.159027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.179489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85115680-a7c9-4412-abfe-96e5d4660c45 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.244066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.244700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.244976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.245246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] PATH_INFO: `/v3/projects/91a6cea77d0f436390ceb0952327fbaa` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.246372 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a6cea77d0f436390ceb0952327fbaa)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.248295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.264145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:47.297319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a6cea77d0f436390ceb0952327fbaa', 'request_id': 'req-e0b8086e-b759-46e9-9dc9-d25eeb480a18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:47.304862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a6cea77d0f436390ceb0952327fbaa', 'request_id': 'req-e0b8086e-b759-46e9-9dc9-d25eeb480a18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:47.311337 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a6cea77d0f436390ceb0952327fbaa', 'request_id': 'req-e0b8086e-b759-46e9-9dc9-d25eeb480a18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:47.322442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] The token cache is being invalidate because project 91a6cea77d0f436390ceb0952327fbaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:02:47.322732 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:47.323393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0b8086e-b759-46e9-9dc9-d25eeb480a18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:47.324888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1245/2520] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:02:47 2022] DELETE /identity/v3/projects/91a6cea77d0f436390ceb0952327fbaa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:47.329704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.329985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.330256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.349912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:47.351650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:47.370774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:47.424652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1276/2521] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:47.432969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.433255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.433536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.453762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:47.455106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:47.463697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1246/2522] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:47.468003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.478984 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1311804-a395-4ed7-bffc-b4309d4eb12f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.483751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.484386 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.484664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.485048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.485700 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:02:47.486003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.487642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.499402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1277/2523] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:02:47.503692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.513098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c8f2d-f712-4182-a63d-fe67317a1fe5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.517739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.518355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.518632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.518908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.519583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.521488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.548831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1247/2524] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:02:47 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:47.552864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.562839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0010598-be65-4a42-af82-ce48a03eb78d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.617694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.618312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.618586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.618850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.619531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.621431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.641926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1278/2525] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:02:47 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:47.645958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.657715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb7aca-bda9-4be6-813d-4bade86b62a4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.663090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.663755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.664037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.664311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.664947 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.666549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.676613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1248/2526] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:02:47.680717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.690548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5df0-bca8-4621-a975-91447521d744 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.695814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efac93c-ad12-4eb6-a974-7f4f742c08fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.696410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efac93c-ad12-4eb6-a974-7f4f742c08fc admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.696692 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efac93c-ad12-4eb6-a974-7f4f742c08fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.696969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efac93c-ad12-4eb6-a974-7f4f742c08fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.697608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efac93c-ad12-4eb6-a974-7f4f742c08fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.699234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efac93c-ad12-4eb6-a974-7f4f742c08fc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.710403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1279/2527] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:02:47.714594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.724819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08104345-c25f-49fc-9c9e-e0e5d46ff152 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.729623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.730296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.730580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.730854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] PATH_INFO: `/v3/projects/a3c8cd9129124bba97b08e1956580d6b/users/25a19bc83f1645cdad7850abf6c57446/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.752432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c8cd9129124bba97b08e1956580d6b, user_id=25a19bc83f1645cdad7850abf6c57446, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.755095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.768624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1249/2528] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:02:47 2022] PUT /identity/v3/projects/a3c8cd9129124bba97b08e1956580d6b/users/25a19bc83f1645cdad7850abf6c57446/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:47.774321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4529f-9cc9-4905-ae63-2ade41bfd3d1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.774778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4529f-9cc9-4905-ae63-2ade41bfd3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.774905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4529f-9cc9-4905-ae63-2ade41bfd3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.793628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7be4529f-9cc9-4905-ae63-2ade41bfd3d1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:47.794829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7be4529f-9cc9-4905-ae63-2ade41bfd3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:47.852172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1280/2529] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:47.861979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:47.863725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:47.919649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd4a85-c77b-44ba-b3d6-ba7cf09f0f37 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:47.923347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:47.923941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:47.924238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:47.924491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:47.933294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:47.934706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:47.944499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1250/2530] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:02:48.583384 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64776265-21ce-466a-aff9-2b080876ed01 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:48.583751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64776265-21ce-466a-aff9-2b080876ed01 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:48.584095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64776265-21ce-466a-aff9-2b080876ed01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:48.611657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-64776265-21ce-466a-aff9-2b080876ed01 None None] MFA Rules not processed for user `25a19bc83f1645cdad7850abf6c57446`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:48.612861 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-64776265-21ce-466a-aff9-2b080876ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:48.633304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:48.686385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1281/2531] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:02:48.698533 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:48.700173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:48.753629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:48.770827 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40421b-bf56-4800-8622-7d299b82492f service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:48.774998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:48.775681 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:48.775950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:48.776217 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:48.784475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:48.785893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:48.794549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1251/2532] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:02:50.214881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bd299-1bd3-4570-9ccd-090cc74629bf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:50.215483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bd299-1bd3-4570-9ccd-090cc74629bf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:50.215483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bd299-1bd3-4570-9ccd-090cc74629bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:50.234412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-dc0bd299-1bd3-4570-9ccd-090cc74629bf None None] MFA Rules not processed for user `25a19bc83f1645cdad7850abf6c57446`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:50.235648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dc0bd299-1bd3-4570-9ccd-090cc74629bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:50.242307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1282/2533] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:50.494626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:50.496080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:50.510207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68fee2-bfd1-43b3-834c-9174dbfc4f1c service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:50.515032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:50.515677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:50.516026 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:50.516235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] PATH_INFO: `/v3/users/d0f67a1377e84c4cabc0ee56f6313980` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:50.517331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f67a1377e84c4cabc0ee56f6313980)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:50.519271 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:50.566542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f67a1377e84c4cabc0ee56f6313980'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:50.574946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f67a1377e84c4cabc0ee56f6313980'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:50.575574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f67a1377e84c4cabc0ee56f6313980'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:50.589130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f67a1377e84c4cabc0ee56f6313980'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:50.596369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f67a1377e84c4cabc0ee56f6313980'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:50.604081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1252/2534] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:02:50 2022] DELETE /identity/v3/users/d0f67a1377e84c4cabc0ee56f6313980 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:50.617847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:50.617847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:50.617847 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:50.628661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:50.629886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:50.693627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1283/2535] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:02:50.701605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:50.703134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:50.769883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d04e4f-c9ee-4891-b325-102a6e6b897d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:50.773546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5176cef-3dd3-470f-a1e5-74556976d7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:50.774147 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176cef-3dd3-470f-a1e5-74556976d7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:50.774426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176cef-3dd3-470f-a1e5-74556976d7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:50.774695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176cef-3dd3-470f-a1e5-74556976d7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:50.783757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5176cef-3dd3-470f-a1e5-74556976d7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:50.785159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5176cef-3dd3-470f-a1e5-74556976d7c0 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:50.795126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1253/2536] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:02:51.088470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.090318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:51.119458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78598-2258-4314-ad91-5026c32cc825 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.124115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.124749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.125032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.125319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] PATH_INFO: `/v3/projects/607ab875b1054309b32f91e7ca2515e9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.126532 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=607ab875b1054309b32f91e7ca2515e9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.128477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.181944 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '607ab875b1054309b32f91e7ca2515e9', 'request_id': 'req-fde10086-4ee2-42ff-a896-7b9a2a300935'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:51.190946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '607ab875b1054309b32f91e7ca2515e9', 'request_id': 'req-fde10086-4ee2-42ff-a896-7b9a2a300935'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:51.197625 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '607ab875b1054309b32f91e7ca2515e9', 'request_id': 'req-fde10086-4ee2-42ff-a896-7b9a2a300935'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:51.209739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] The token cache is being invalidate because project 607ab875b1054309b32f91e7ca2515e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:02:51.210060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:51.210678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fde10086-4ee2-42ff-a896-7b9a2a300935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:51.212339 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1284/2537] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:02:51 2022] DELETE /identity/v3/projects/607ab875b1054309b32f91e7ca2515e9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:02:51.217680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.218064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.218394 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.241800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:51.243021 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:51.297842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:51.308177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1254/2538] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:51.316975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.317472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.317570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.337607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:51.338817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:51.347049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1285/2539] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:02:51.351352 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.361592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c9a8e-63c4-408a-9fa9-ee5fc393724c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.367461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.368077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.368978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.369258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.369972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:02:51.370285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.372235 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.385950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1255/2540] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:02:51.390469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.399953 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b539c-8686-42d4-91ae-44498f0ad6d1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.404595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.405387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.405818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.406270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.406924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.408881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.435808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1286/2541] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:02:51 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:51.439854 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.450396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743629d-b780-4dd4-840f-085e74579b0d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.513731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.514436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.514606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.514901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.515572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.518715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.544479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1256/2542] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:02:51 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:02:51.545712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.557179 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59373b11-bdc4-465c-ba77-b98886eab1c8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.562423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.563034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.563350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.563627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.564270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.566010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.568266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:51.577605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1287/2543] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:02:51.581808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.591748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6069559-2a9a-4b0a-a3ab-471e4d8f8229 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.596456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f8164-9bc6-417b-b1c7-8876d3cf3433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.597057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f8164-9bc6-417b-b1c7-8876d3cf3433 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.597340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f8164-9bc6-417b-b1c7-8876d3cf3433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.597620 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f8164-9bc6-417b-b1c7-8876d3cf3433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.598259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f8164-9bc6-417b-b1c7-8876d3cf3433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.600780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f8164-9bc6-417b-b1c7-8876d3cf3433 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.612179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1257/2544] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:02:51.616843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.626996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b656c-c00a-4165-bea4-ae12ef6fdba6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.632008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.632687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.633246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.633340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] PATH_INFO: `/v3/projects/7ade45dd41b848c5801b6d37a4232f86/users/945b92d7096c456884ff41bad41bb0da/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.649575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ade45dd41b848c5801b6d37a4232f86, user_id=945b92d7096c456884ff41bad41bb0da, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.652495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.670557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1288/2545] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:02:51 2022] PUT /identity/v3/projects/7ade45dd41b848c5801b6d37a4232f86/users/945b92d7096c456884ff41bad41bb0da/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:51.675422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89babf4-5835-42dc-b3be-bb905e11f7d3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.675713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89babf4-5835-42dc-b3be-bb905e11f7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.675989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89babf4-5835-42dc-b3be-bb905e11f7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.704424 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c89babf4-5835-42dc-b3be-bb905e11f7d3 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:51.705641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c89babf4-5835-42dc-b3be-bb905e11f7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:51.769133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1258/2546] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:02:51.781403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:51.783342 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:51.840776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbd02d-5594-4f32-a5d8-0ec92b25678a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:51.844364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:51.844969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:51.845247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:51.845517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:51.855014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:51.856482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:51.865811 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1289/2547] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:02:56.359885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.359885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.359885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.390155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] MFA Rules not processed for user `945b92d7096c456884ff41bad41bb0da`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:56.391409 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:56.448274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:56.459428 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1259/2548] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:56.463740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.473553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb5ffd-b447-489b-9838-bfc5c1027989 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:56.479049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:56.479701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.480034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.480311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.480957 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:56.482880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:56.512253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1290/2549] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:02:56 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:56.516354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.539478 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7133c5-a9f7-4c31-bd89-874ad2bb78d6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:56.613667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:56.614287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.614570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.614845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.616249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:56.618188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:56.640561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1260/2550] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:02:56 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:02:56.644691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.656855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077ce18-ca58-40e4-9cdf-2969b452e37f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:56.661971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:56.663140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.663533 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.663900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.664622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:56.666483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:56.686181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1291/2551] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:02:56.687406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.701320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ec61-5a91-43e6-b5c8-799deca20686 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:56.707457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:56.708318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.708761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.709178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] PATH_INFO: `/v3/projects/6ae921f7a2cb48aaa698c627874b544e/users/188e5c33eff64b1c8893574ef358b107/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.733091 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae921f7a2cb48aaa698c627874b544e, user_id=188e5c33eff64b1c8893574ef358b107, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:56.736023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:56.762539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1261/2552] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:02:56 2022] PUT /identity/v3/projects/6ae921f7a2cb48aaa698c627874b544e/users/188e5c33eff64b1c8893574ef358b107/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:56.764016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.775398 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e63f9-a38c-44f7-bd48-a008b924358d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:56.839803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017d91-0a3b-45ad-b9ba-8baa769027ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:56.840544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017d91-0a3b-45ad-b9ba-8baa769027ca admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.840887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017d91-0a3b-45ad-b9ba-8baa769027ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.841205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017d91-0a3b-45ad-b9ba-8baa769027ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.842208 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017d91-0a3b-45ad-b9ba-8baa769027ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:56.844115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017d91-0a3b-45ad-b9ba-8baa769027ca admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:56.856587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1292/2553] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:02:56.860967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.874188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e131-3ef6-43cd-a239-0ebb0023a348 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:56.880452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:56.881346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:56.881625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:56.881892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] PATH_INFO: `/v3/projects/6ae921f7a2cb48aaa698c627874b544e/users/188e5c33eff64b1c8893574ef358b107/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:56.884068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae921f7a2cb48aaa698c627874b544e, user_id=188e5c33eff64b1c8893574ef358b107, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:56.887251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:56.898180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1262/2554] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:02:56 2022] PUT /identity/v3/projects/6ae921f7a2cb48aaa698c627874b544e/users/188e5c33eff64b1c8893574ef358b107/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:02.404233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa8724-1a3b-4348-9dec-c807adb4c1f9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:02.404781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa8724-1a3b-4348-9dec-c807adb4c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:02.404781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa8724-1a3b-4348-9dec-c807adb4c1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:02.436367 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f7fa8724-1a3b-4348-9dec-c807adb4c1f9 None None] MFA Rules not processed for user `188e5c33eff64b1c8893574ef358b107`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:02.437500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f7fa8724-1a3b-4348-9dec-c807adb4c1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:02.468528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:02.513958 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1293/2555] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:02.526698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:02.528470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:02.585445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799bac5-9c78-4dcc-a113-ceff6001f040 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:02.589055 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:02.589658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:02.589934 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:02.590208 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:02.669086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:02.670662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:02.682473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1263/2556] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:02.861047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764a85-7108-424d-8029-d5c375e5832b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:02.861047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764a85-7108-424d-8029-d5c375e5832b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:02.861047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23764a85-7108-424d-8029-d5c375e5832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:02.901963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-23764a85-7108-424d-8029-d5c375e5832b None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:02.907344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-23764a85-7108-424d-8029-d5c375e5832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:02.981918 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1294/2557] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:02.994016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:02.995882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:03.001286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:03.075876 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93c1af-7695-4f4d-b545-5eac876ed4e9 service glance] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:03.080963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:03.081525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:03.081975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:03.082252 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:03.091792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:03.093433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:03.102844 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1264/2558] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:03:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 15:03:06.749803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:06.750401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:06.750599 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:06.779318 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:06.780784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:06.789531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1295/2559] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:03:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:06.815158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:06.827546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fc12d-18db-4117-9abe-9b3116ab0a0b service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:06.832959 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdeae0-92a6-4f44-9b33-6d6bdd62e8c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:06.834230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdeae0-92a6-4f44-9b33-6d6bdd62e8c0 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:06.834576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdeae0-92a6-4f44-9b33-6d6bdd62e8c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:06.834924 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdeae0-92a6-4f44-9b33-6d6bdd62e8c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:06.844286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdeae0-92a6-4f44-9b33-6d6bdd62e8c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:06.845966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdeae0-92a6-4f44-9b33-6d6bdd62e8c0 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:06.858846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1265/2560] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:03:13.165505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:13.166875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:13.252883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abded4e7-aa72-48a1-bbad-66b1731089fb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:13.258952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:13.261494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:13.262014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:13.263094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:13.265287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:13.396685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:13.398545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:13.410919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1296/2561] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:13.803766 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9736d28-3aa5-48ee-9c1d-7081af60dff7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:13.804304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9736d28-3aa5-48ee-9c1d-7081af60dff7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:13.804515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9736d28-3aa5-48ee-9c1d-7081af60dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:13.829974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b9736d28-3aa5-48ee-9c1d-7081af60dff7 None None] MFA Rules not processed for user `945b92d7096c456884ff41bad41bb0da`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:13.831228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b9736d28-3aa5-48ee-9c1d-7081af60dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:13.851717 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1266/2562] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:03:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:14.635954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:14.641442 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:14.645147 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d57e0-fea7-46e4-99f3-56f7bd10583e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:14.651373 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:14.652026 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:14.652558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:14.652558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:14.654309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:14.669437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:14.670890 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:14.682933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1297/2563] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:22.388728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:22.389327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:22.389327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:22.409565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:22.411188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:22.420506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1267/2564] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:22.440005 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:22.455056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f84d-fa89-490c-a16d-de8ac27ec72b service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:22.460868 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:22.461559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:22.461907 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:22.462254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:22.475522 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:22.477178 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:22.487343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1298/2565] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:03:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 15:03:24.333363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:24.334842 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:24.350507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483eb3b-e1de-4199-85c4-b95e915514ac None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:24.355988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:24.356582 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:24.356861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:24.357133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:24.358901 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:24.440364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:24.442001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a9d82-28b1-425b-b00f-963b4a108e30 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:24.451767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1268/2566] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:32.482541 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=10 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:34.447531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=71 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:40.567673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:40.581451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56753b-d4c6-4625-b28e-15982555454a service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:40.586971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:40.592404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.593132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.593132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.595120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:40.608773 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:40.610284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62144b6-dcb0-4376-b04a-4a6a700e21ac service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:40.619223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1299/2567] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:40.755858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8e51-dd4d-4fce-9e25-40f2a459debb None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.756651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8e51-dd4d-4fce-9e25-40f2a459debb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.757243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad8e51-dd4d-4fce-9e25-40f2a459debb None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.759403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1269/2568] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 15:03:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:03:40.765383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4a825-2d8f-4591-a29a-22369f4369bc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.765676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4a825-2d8f-4591-a29a-22369f4369bc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.765963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4a825-2d8f-4591-a29a-22369f4369bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.790143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-37e4a825-2d8f-4591-a29a-22369f4369bc None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:40.791483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-37e4a825-2d8f-4591-a29a-22369f4369bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:40.801324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1300/2569] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 15:03:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:40.813908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8e51-dd4d-4fce-9e25-40f2a459debb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:40.815724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1aad8e51-dd4d-4fce-9e25-40f2a459debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:40.830154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad8e51-dd4d-4fce-9e25-40f2a459debb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:40.833740 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1565b17-5b4a-4758-9cd7-5c09fc8e3eb4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:40.834370 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1565b17-5b4a-4758-9cd7-5c09fc8e3eb4 service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.834645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1565b17-5b4a-4758-9cd7-5c09fc8e3eb4 service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.835113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1565b17-5b4a-4758-9cd7-5c09fc8e3eb4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.845595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1565b17-5b4a-4758-9cd7-5c09fc8e3eb4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:40.847351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1565b17-5b4a-4758-9cd7-5c09fc8e3eb4 service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:40.860856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1270/2570] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:40.938254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dd000-7471-4c46-a880-ba4d6f0a1706 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.938639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dd000-7471-4c46-a880-ba4d6f0a1706 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.938998 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dd000-7471-4c46-a880-ba4d6f0a1706 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.940535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1301/2571] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 15:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:03:40.945882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da5936-7ac3-4f88-8dff-408b4b9be1ea None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.946186 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da5936-7ac3-4f88-8dff-408b4b9be1ea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.946448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da5936-7ac3-4f88-8dff-408b4b9be1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.966060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-45da5936-7ac3-4f88-8dff-408b4b9be1ea None None] MFA Rules not processed for user `60c809b5a47a4983be26116553934387`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:40.967288 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-45da5936-7ac3-4f88-8dff-408b4b9be1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:40.974378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1271/2572] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 15:03:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:40.986377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dd000-7471-4c46-a880-ba4d6f0a1706 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:40.997666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dd000-7471-4c46-a880-ba4d6f0a1706 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.001786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a521529-d516-4c55-88ea-abb488a937b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.002460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a521529-d516-4c55-88ea-abb488a937b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.002834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a521529-d516-4c55-88ea-abb488a937b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.003173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a521529-d516-4c55-88ea-abb488a937b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.011631 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a521529-d516-4c55-88ea-abb488a937b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.013116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a521529-d516-4c55-88ea-abb488a937b0 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.021772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1302/2573] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:41.101415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1445d-52e3-445b-a34d-4fc99323d259 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.101709 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1445d-52e3-445b-a34d-4fc99323d259 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.101989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1445d-52e3-445b-a34d-4fc99323d259 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.103430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1272/2574] 10.222.0.66 () {54 vars in 921 bytes} [Thu Sep 22 15:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:03:41.108094 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.108380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.108649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.127927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] MFA Rules not processed for user `ff1a6a40166e4f51a14cca9cd510e680`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:41.129649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:41.140108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1303/2575] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Sep 22 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:41.154331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1445d-52e3-445b-a34d-4fc99323d259 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.166244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1445d-52e3-445b-a34d-4fc99323d259 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.169835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.170430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.170708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.170981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.184927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.186384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.197698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1273/2576] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:41.287295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.288711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:41.356397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cd2ff-b371-49fe-9185-0f90e6081989 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.361322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.361917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.362194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.362465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.364719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:41.375747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.377330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.387809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1304/2577] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:43.350968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.362788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792109b-f54f-415d-afbc-114a76b97887 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.368972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.369573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.369852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.370125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.371928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:43.431963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.433722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485c1e8-3529-4306-9727-f2622b6aa60e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.443362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1274/2578] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:51.336877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:51.346531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2dcb2-5ca1-4597-bbf6-c31023cf1917 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:51.350956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:51.351594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:51.351876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:51.352149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:51.353910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:51.363497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:51.364931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:51.372970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1305/2579] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:53.438636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=43 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:55.457632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:55.458289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:55.458289 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:55.479482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:55.480680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:55.491219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1275/2580] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:55.503257 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:55.515798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b3d4a-f067-4927-8bdd-2954bf58cc70 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:55.521443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:55.522074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:55.522356 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:55.522646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:55.530881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:55.532559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:55.542438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1306/2581] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:03:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:04:05.473661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:05.538795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=62 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:24.095383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:24.112487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:24.113561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4b9b1-38e9-4e22-b58a-e85d22155f49 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:24.120678 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:24.121995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:24.122259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:24.122865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:24.126160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:24.131997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31975673-5861-47e5-a868-3250e22ba851 service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:24.136874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:24.137493 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:24.137825 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:24.138103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:24.147722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:24.147938 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:24.149308 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:24.149420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:24.159569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1307/2582] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:24.161174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1276/2583] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:25.865295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:25.889114 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ac5c1-2332-48e6-b2d1-f88634829b2c service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:25.897676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:25.898573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:25.898862 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:25.899162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:25.901105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:25.915870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:25.917540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c698fb-ff70-4dc8-b197-e51e6a3588f4 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:25.930446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1308/2584] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:27.560180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:27.562019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:27.577273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb79d6-9300-484f-844e-4120bc1dafbf service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:27.584231 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:27.584853 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:27.585134 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:27.585408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:27.587163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:27.600066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:27.601802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:27.617468 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1277/2585] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:28.622672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a0a51-2051-4e71-a2a4-f6fc6c2f2517 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:28.623124 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a0a51-2051-4e71-a2a4-f6fc6c2f2517 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:28.623230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a0a51-2051-4e71-a2a4-f6fc6c2f2517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:28.643628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-686a0a51-2051-4e71-a2a4-f6fc6c2f2517 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:28.645477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-686a0a51-2051-4e71-a2a4-f6fc6c2f2517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:28.654157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1309/2586] 10.222.0.66 () {58 vars in 967 bytes} [Thu Sep 22 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:28.680430 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:28.697882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea3f58-3c8d-4609-94ea-1d86cb4add9f service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:28.703314 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101465ab-f0ed-43ad-8e55-0e9114bbeffe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:28.704128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101465ab-f0ed-43ad-8e55-0e9114bbeffe service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:28.704418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101465ab-f0ed-43ad-8e55-0e9114bbeffe service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:28.704693 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101465ab-f0ed-43ad-8e55-0e9114bbeffe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:28.714077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101465ab-f0ed-43ad-8e55-0e9114bbeffe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:28.715692 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101465ab-f0ed-43ad-8e55-0e9114bbeffe service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:28.725846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1278/2587] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:04:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:04:38.638246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=15 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:38.720557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=18 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:43.094184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:43.094762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:43.094762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:43.113867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] MFA Rules not processed for user `25a19bc83f1645cdad7850abf6c57446`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:43.115067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:43.170116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1310/2588] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:45.013896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:45.014420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:45.014420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:45.033320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] MFA Rules not processed for user `dab1b8b77cfb479098ac9e0d785a25c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:45.034539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:45.042043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1279/2589] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:53.163544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=25 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:55.028506 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:56.337451 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:56.338959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:56.403876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f244d4-fb2c-4c6b-8c9b-08680e043856 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:56.408379 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:56.409003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:56.409281 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:56.409552 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] PATH_INFO: `/v3/users/25a19bc83f1645cdad7850abf6c57446` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:56.410755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a19bc83f1645cdad7850abf6c57446)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:56.412785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:56.458181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a19bc83f1645cdad7850abf6c57446'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.465685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a19bc83f1645cdad7850abf6c57446'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.478204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a19bc83f1645cdad7850abf6c57446'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.478916 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a19bc83f1645cdad7850abf6c57446'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.479633 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d5627b69-d6fb-4882-8675-f1c826ca2928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a19bc83f1645cdad7850abf6c57446'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.492664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1311/2590] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:56 2022] DELETE /identity/v3/users/25a19bc83f1645cdad7850abf6c57446 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:56.657647 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:56.669001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d162b9-2732-4e1b-977d-3717c7c07daa None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:56.726952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:56.727595 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:56.727870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:56.728135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] PATH_INFO: `/v3/projects/a3c8cd9129124bba97b08e1956580d6b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:56.729254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c8cd9129124bba97b08e1956580d6b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:56.731141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:56.906612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c8cd9129124bba97b08e1956580d6b', 'request_id': 'req-9e803bcf-5714-45bb-a999-989f3c2ff2a4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.914166 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c8cd9129124bba97b08e1956580d6b', 'request_id': 'req-9e803bcf-5714-45bb-a999-989f3c2ff2a4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.919594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c8cd9129124bba97b08e1956580d6b', 'request_id': 'req-9e803bcf-5714-45bb-a999-989f3c2ff2a4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.930412 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] The token cache is being invalidate because project a3c8cd9129124bba97b08e1956580d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:56.930700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.931359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:56.932832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1280/2591] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:56 2022] DELETE /identity/v3/projects/a3c8cd9129124bba97b08e1956580d6b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:06.488333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:06.927598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:11.500169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:11.500716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:11.500716 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:11.530665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] MFA Rules not processed for user `2febff9d4cbb476c90037ffacf44465d`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:11.532001 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:11.606121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1312/2592] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:21.604348 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=37 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:28.408884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:28.410319 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:28.488838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e803bcf-5714-45bb-a999-989f3c2ff2a4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:28.492486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:28.493140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:28.493362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:28.493636 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:28.495407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:28.587589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:28.589014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:28.608127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1281/2593] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:32.155875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:32.157399 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:32.216981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:32.218452 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:32.236982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d15877-0c52-4d84-a2cb-1b4c2fb288a9 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:32.241670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:32.242302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.243128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.243624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] PATH_INFO: `/v3/users/dab1b8b77cfb479098ac9e0d785a25c6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.244967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab1b8b77cfb479098ac9e0d785a25c6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:32.246996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:32.271354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3ae81-f4b5-41da-a751-1e8fec3eeef1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:32.277210 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:32.277878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.278177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.278514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.280981 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:32.294848 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:32.296485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:32.298935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab1b8b77cfb479098ac9e0d785a25c6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.301385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:32.307733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab1b8b77cfb479098ac9e0d785a25c6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.308402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab1b8b77cfb479098ac9e0d785a25c6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.309164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1313/2594] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:32.322374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab1b8b77cfb479098ac9e0d785a25c6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.330039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3739ddeb-3a31-4b30-943c-71c8b01f5fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab1b8b77cfb479098ac9e0d785a25c6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.336605 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1282/2595] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:32 2022] DELETE /identity/v3/users/dab1b8b77cfb479098ac9e0d785a25c6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:32.511015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:32.521928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf80809-5e3c-458f-bb95-3f12683cf46c service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:32.578905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:32.579573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.579841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.580109 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] PATH_INFO: `/v3/projects/6edd1ecc737c48c3b26fa7dea7a48ddd` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.581207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6edd1ecc737c48c3b26fa7dea7a48ddd)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:32.583112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:32.627789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6edd1ecc737c48c3b26fa7dea7a48ddd', 'request_id': 'req-67add920-1d0a-4b2a-9172-5466baf0faf8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.635790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6edd1ecc737c48c3b26fa7dea7a48ddd', 'request_id': 'req-67add920-1d0a-4b2a-9172-5466baf0faf8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.641292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6edd1ecc737c48c3b26fa7dea7a48ddd', 'request_id': 'req-67add920-1d0a-4b2a-9172-5466baf0faf8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.648701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:32.652711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] The token cache is being invalidate because project 6edd1ecc737c48c3b26fa7dea7a48ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:32.653336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.654162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-67add920-1d0a-4b2a-9172-5466baf0faf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:32.655751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1314/2596] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:32 2022] DELETE /identity/v3/projects/6edd1ecc737c48c3b26fa7dea7a48ddd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:32.666177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.666456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.666726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.721874 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:32.722972 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:32.780615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1283/2597] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:32.789926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.790207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.790481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.810615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:32.811831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:32.819619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1315/2598] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:32.823883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:32.834303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e181d17-bf31-4567-9caa-65649677b923 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:32.839007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:32.839710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.840056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.840397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.841108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:32.841479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:32.843187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:32.856107 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1284/2599] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:06:32.860425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:32.871726 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b096798-8e41-4272-98cc-071a73a87384 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:32.876509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:32.877223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:32.877593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:32.877937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:32.878654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:32.880688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:32.916344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1316/2600] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:32 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:32.917498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:32.945137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82041af-7acc-4627-905e-95073f782f8b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.004031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.004556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.004832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.005105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.005722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.007659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.028488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1285/2601] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:32 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:33.032816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.043788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8879b-c903-4c68-a2ad-b3bcba27b648 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.048351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.048943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.049224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.049497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.050161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.051791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.062248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1317/2602] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:33.066198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.076025 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536176ba-929f-45fa-b05f-6aee14274a0d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.080856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdce1e0-9610-44f2-aa31-3d56d077926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.081503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdce1e0-9610-44f2-aa31-3d56d077926a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.081780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdce1e0-9610-44f2-aa31-3d56d077926a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.082045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdce1e0-9610-44f2-aa31-3d56d077926a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.082686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdce1e0-9610-44f2-aa31-3d56d077926a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.084286 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdce1e0-9610-44f2-aa31-3d56d077926a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.094212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1286/2603] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:33.098164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.108409 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972e78-7fc0-4c8e-8bab-e61fcf014c74 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.112975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.113606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.113886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.114162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] PATH_INFO: `/v3/projects/b2bfed9cad7347f79c15df144a770fda/users/0b7b887096b44b36b0079d092c9d9be4/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.129195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2bfed9cad7347f79c15df144a770fda, user_id=0b7b887096b44b36b0079d092c9d9be4, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.131886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.144702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1318/2604] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:33 2022] PUT /identity/v3/projects/b2bfed9cad7347f79c15df144a770fda/users/0b7b887096b44b36b0079d092c9d9be4/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:33.154951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.155272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.155550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.163346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:33.188104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] MFA Rules not processed for user `0b7b887096b44b36b0079d092c9d9be4`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:33.189298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:33.256119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1287/2605] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:33.260876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.321363 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3502cc-0a86-4602-8ac9-1edf0e49e15f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.326014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.326614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.326893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.327167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.327819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.329711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.356216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1319/2606] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:33.360261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.370883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879535-1c14-4d1a-9bd0-4aec1cf96288 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.425907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.426500 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.426801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.427067 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.427720 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.429639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.450661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1288/2607] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:33 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:33.460137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.471802 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db4c4-9918-4fb9-872a-7e7b6fc139ab admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.476432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.477733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.478103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.478378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.479018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.483243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.492506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1320/2608] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:33.496469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.506118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852dd99-786d-469a-88a8-c20f1d84f5f0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.510938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af7f76-38bd-4fcc-b488-9cd1250f7404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.511495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af7f76-38bd-4fcc-b488-9cd1250f7404 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.511788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af7f76-38bd-4fcc-b488-9cd1250f7404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.512054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af7f76-38bd-4fcc-b488-9cd1250f7404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.512683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af7f76-38bd-4fcc-b488-9cd1250f7404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.514239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af7f76-38bd-4fcc-b488-9cd1250f7404 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.524516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1289/2609] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:33.528283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.538138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60b589-f17b-468e-94d4-04731b83b09b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.543600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.544273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.544575 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.544851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] PATH_INFO: `/v3/projects/1002bf1c1c7c47c586c917a13246a57b/users/59808e0497d44226a3ef861bdab864e0/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.564117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1002bf1c1c7c47c586c917a13246a57b, user_id=59808e0497d44226a3ef861bdab864e0, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.566851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.580647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1321/2610] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:33 2022] PUT /identity/v3/projects/1002bf1c1c7c47c586c917a13246a57b/users/59808e0497d44226a3ef861bdab864e0/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:33.591487 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.591948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.592250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.621857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] MFA Rules not processed for user `59808e0497d44226a3ef861bdab864e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:33.623047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:33.684155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:33.688607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1290/2611] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:33.692863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.740335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:33.754810 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d4af8-194c-45c4-8332-e5fe481d70ef admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.759937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.760530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.760808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.761128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.761689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.763635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.789327 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1322/2612] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:33 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:33.793354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.804101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbe632-7f2b-4ad5-8089-0d2218998ae1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.865617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.866238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.866521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.866795 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.868559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.870576 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.896896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1291/2613] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:33 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:33.901932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.912988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab265-5dab-4883-b234-ae6b26ea0945 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.917987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.918577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.918853 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.919120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.919781 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.921366 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.932254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1323/2614] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:33.936381 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.946419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5994-6dc6-49b4-a923-4657bc5f8df0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:33.951062 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:33.951765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:33.952046 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:33.952317 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] PATH_INFO: `/v3/projects/5121db0a3d1943e8b225a89c21e940e6/users/ef5e480fb242436cb2c4a27658654406/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:33.967256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5121db0a3d1943e8b225a89c21e940e6, user_id=ef5e480fb242436cb2c4a27658654406, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:33.969954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:33.983694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1292/2615] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:33 2022] PUT /identity/v3/projects/5121db0a3d1943e8b225a89c21e940e6/users/ef5e480fb242436cb2c4a27658654406/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:33.987590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:33.997673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a18d8-3112-4b6f-b069-37d5c2341626 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:34.053069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195825cf-2d31-492a-817d-9ac78efe5203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:34.053665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195825cf-2d31-492a-817d-9ac78efe5203 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:34.053946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195825cf-2d31-492a-817d-9ac78efe5203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:34.054221 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195825cf-2d31-492a-817d-9ac78efe5203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:34.054867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195825cf-2d31-492a-817d-9ac78efe5203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:34.056499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195825cf-2d31-492a-817d-9ac78efe5203 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:34.073468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1324/2616] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:34.077714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:34.088247 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17cfbf-1668-4790-a1e0-b644aaa80dfb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:34.094777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:34.095448 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:34.095730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:34.096001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] PATH_INFO: `/v3/projects/5121db0a3d1943e8b225a89c21e940e6/users/ef5e480fb242436cb2c4a27658654406/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:34.097893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5121db0a3d1943e8b225a89c21e940e6, user_id=ef5e480fb242436cb2c4a27658654406, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:34.100554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:34.111080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1293/2617] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:34 2022] PUT /identity/v3/projects/5121db0a3d1943e8b225a89c21e940e6/users/ef5e480fb242436cb2c4a27658654406/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:34.121025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:34.121385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:34.121734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:34.150112 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] MFA Rules not processed for user `ef5e480fb242436cb2c4a27658654406`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:34.151355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:34.223778 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1325/2618] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:34.234597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:34.236323 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:34.293014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c7c9f-de3e-496d-8746-ababc0f1fc3e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:34.296646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:34.297232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:34.297510 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:34.297778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:34.307273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:34.308641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:34.317746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1294/2619] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:34.797354 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:34.798840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:34.811432 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed8d05-7deb-41e3-8195-6d69c5462c30 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:34.815057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:34.815688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:34.815971 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:34.816243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:34.856725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:34.873919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:34.875404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:34.884836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1326/2620] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:38.902503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:38.920589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c508695-f2d5-4ba7-9c0b-dc7202e9e40c service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:38.924346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79cdec-664c-47b4-bad1-ddfa71039472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:38.924938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79cdec-664c-47b4-bad1-ddfa71039472 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:38.925407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79cdec-664c-47b4-bad1-ddfa71039472 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:38.925676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79cdec-664c-47b4-bad1-ddfa71039472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:38.972299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:38.985699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79cdec-664c-47b4-bad1-ddfa71039472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:38.987063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79cdec-664c-47b4-bad1-ddfa71039472 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:38.995818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1295/2621] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:41.078654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:41.139676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8f115-600a-4bca-9f8a-977df3d77a58 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:41.144153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:41.144771 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:41.145050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:41.145323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] PATH_INFO: `/v3/users/0b7b887096b44b36b0079d092c9d9be4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:41.146429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7b887096b44b36b0079d092c9d9be4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:41.148978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:41.193013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7b887096b44b36b0079d092c9d9be4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.201169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7b887096b44b36b0079d092c9d9be4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.214489 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7b887096b44b36b0079d092c9d9be4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.215152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7b887096b44b36b0079d092c9d9be4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.215785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7b887096b44b36b0079d092c9d9be4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.228176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1327/2622] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:41 2022] DELETE /identity/v3/users/0b7b887096b44b36b0079d092c9d9be4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:41.232564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:41.232996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:41.233256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:41.253405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:41.254702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:41.313058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1296/2623] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:41.322896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:41.324365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:41.388600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:41.403661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458be53-e801-4665-b685-f5eeea281711 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:41.407374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:41.407978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:41.408258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:41.408529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:41.417943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:41.419470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:41.428719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1328/2624] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:41.694671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:41.696626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:41.708460 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c361-794d-4c69-a9ce-f3a4071e6514 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:41.713158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:41.713788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:41.714072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:41.714340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] PATH_INFO: `/v3/projects/b2bfed9cad7347f79c15df144a770fda` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:41.715517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2bfed9cad7347f79c15df144a770fda)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:41.717517 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:41.763985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2bfed9cad7347f79c15df144a770fda', 'request_id': 'req-96e3042c-b691-4a71-9316-bf68a9411ef3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.770792 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2bfed9cad7347f79c15df144a770fda', 'request_id': 'req-96e3042c-b691-4a71-9316-bf68a9411ef3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.776495 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2bfed9cad7347f79c15df144a770fda', 'request_id': 'req-96e3042c-b691-4a71-9316-bf68a9411ef3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.787302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] The token cache is being invalidate because project b2bfed9cad7347f79c15df144a770fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:41.787591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.788212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.790152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1297/2625] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:41 2022] DELETE /identity/v3/projects/b2bfed9cad7347f79c15df144a770fda => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:41.794319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:41.803889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77b31-e915-4ff2-a522-d793820eaff5 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:41.859499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:41.860054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:41.860334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:41.860605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] PATH_INFO: `/v3/users/59808e0497d44226a3ef861bdab864e0` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:41.861708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59808e0497d44226a3ef861bdab864e0)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:41.863705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:41.909940 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59808e0497d44226a3ef861bdab864e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.917567 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59808e0497d44226a3ef861bdab864e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.930186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59808e0497d44226a3ef861bdab864e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.930834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59808e0497d44226a3ef861bdab864e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.931487 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59808e0497d44226a3ef861bdab864e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:41.943557 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1329/2626] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:41 2022] DELETE /identity/v3/users/59808e0497d44226a3ef861bdab864e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:42.216521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:42.218066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:42.237534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3042c-b691-4a71-9316-bf68a9411ef3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:42.297446 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:42.298100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:42.298380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:42.298649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] PATH_INFO: `/v3/projects/1002bf1c1c7c47c586c917a13246a57b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:42.299775 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1002bf1c1c7c47c586c917a13246a57b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:42.301670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:42.311613 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:42.348808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1002bf1c1c7c47c586c917a13246a57b', 'request_id': 'req-98a7a0a1-217d-4a83-bc64-bb0053a8524a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.356094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1002bf1c1c7c47c586c917a13246a57b', 'request_id': 'req-98a7a0a1-217d-4a83-bc64-bb0053a8524a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.361496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1002bf1c1c7c47c586c917a13246a57b', 'request_id': 'req-98a7a0a1-217d-4a83-bc64-bb0053a8524a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.372137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] The token cache is being invalidate because project 1002bf1c1c7c47c586c917a13246a57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:42.372447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.373078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.374529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1298/2627] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:42 2022] DELETE /identity/v3/projects/1002bf1c1c7c47c586c917a13246a57b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:42.378434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:42.379891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:42.392974 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8006b0-29df-442e-ae57-981a92d4c155 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:42.448264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:42.448892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:42.449217 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:42.449488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] PATH_INFO: `/v3/users/ef5e480fb242436cb2c4a27658654406` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:42.450597 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5e480fb242436cb2c4a27658654406)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:42.452549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:42.454785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:42.501595 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5e480fb242436cb2c4a27658654406'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.509475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5e480fb242436cb2c4a27658654406'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.521360 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5e480fb242436cb2c4a27658654406'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.522034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5e480fb242436cb2c4a27658654406'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.522661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-fbf1d38e-f42c-42b8-99ed-42c3d286bacc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5e480fb242436cb2c4a27658654406'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.535729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1330/2628] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:42 2022] DELETE /identity/v3/users/ef5e480fb242436cb2c4a27658654406 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:42.807154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:42.808665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:42.825039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7a0a1-217d-4a83-bc64-bb0053a8524a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:42.880789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:42.881417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:42.881714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:42.881975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] PATH_INFO: `/v3/projects/5121db0a3d1943e8b225a89c21e940e6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:42.883850 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5121db0a3d1943e8b225a89c21e940e6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:42.885743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:42.930565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5121db0a3d1943e8b225a89c21e940e6', 'request_id': 'req-d73741e4-6e9b-424a-8e20-fb7129bc95c7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.937575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5121db0a3d1943e8b225a89c21e940e6', 'request_id': 'req-d73741e4-6e9b-424a-8e20-fb7129bc95c7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.943020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5121db0a3d1943e8b225a89c21e940e6', 'request_id': 'req-d73741e4-6e9b-424a-8e20-fb7129bc95c7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.954310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] The token cache is being invalidate because project 5121db0a3d1943e8b225a89c21e940e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:42.954585 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.955257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d73741e4-6e9b-424a-8e20-fb7129bc95c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:42.957052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1299/2629] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:42 2022] DELETE /identity/v3/projects/5121db0a3d1943e8b225a89c21e940e6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:42.963396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:42.963685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:42.963963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:42.986227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:42.987632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:43.046553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1331/2630] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:43.054837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.055135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.055423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.074233 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:43.075443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:43.083087 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1300/2631] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:43.087714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.097283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9aa88-0dc0-42cf-a729-9aa332245952 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.102449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.103035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.103345 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.103616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.104287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:43.104587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.106171 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.117454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1332/2632] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:43.121449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.130251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4667c6-9e1d-4a88-bd50-3dd97cd9eb95 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.135504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.136086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.136358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.136626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.137238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.139096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.160275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:43.168011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1301/2633] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:43.172048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.181897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9515-02bf-4291-8267-df62f6abfbc0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.237714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.238538 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.238815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.239083 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.239741 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.241679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.255993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:43.264241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1333/2634] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:43.268162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.278770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133a30-c546-45bb-a68c-01c0408d4981 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.283439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.284029 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.284309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.284578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.285207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.286778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.297017 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1302/2635] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:43.300797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.310546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f87be-6957-45eb-a52b-10e99fd8ee5a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.315224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d6a3a-7e7a-471a-b80e-3d3d437ecce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.315822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6a3a-7e7a-471a-b80e-3d3d437ecce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.316101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6a3a-7e7a-471a-b80e-3d3d437ecce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.316371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d6a3a-7e7a-471a-b80e-3d3d437ecce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.317007 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d6a3a-7e7a-471a-b80e-3d3d437ecce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.318596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d6a3a-7e7a-471a-b80e-3d3d437ecce3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.329744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1334/2636] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:43.334013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.344973 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a15ea-9b50-40c7-8c3e-f5023b0c372f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.351108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.352347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.352756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.353041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] PATH_INFO: `/v3/projects/b71fd98477e04a55aebc7dc2b44c2610/users/75f4fa33abf54bdd9749e75bdc3681a8/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.368811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71fd98477e04a55aebc7dc2b44c2610, user_id=75f4fa33abf54bdd9749e75bdc3681a8, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.371624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.385713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1303/2637] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:43 2022] PUT /identity/v3/projects/b71fd98477e04a55aebc7dc2b44c2610/users/75f4fa33abf54bdd9749e75bdc3681a8/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:43.398030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.399170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.399170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.426483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] MFA Rules not processed for user `75f4fa33abf54bdd9749e75bdc3681a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:43.427723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:43.490529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1335/2638] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:43.495461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.555674 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546abcd0-03dd-4010-a69d-e42c0741766c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.560467 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.561063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.561347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.561623 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.562259 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.564507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.591696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1304/2639] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:43.595762 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.606397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2e635-37e7-436a-9576-15873d360f09 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.661849 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.662450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.662742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.662996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.663646 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.665591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.685959 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1336/2640] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:43.689990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.700960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc40c9-924f-4fe5-9757-6b4ebf482b05 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.705511 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.706099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.706377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.706648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.707309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.708884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.718979 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1305/2641] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:43.722838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.732732 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671427d-9b00-418a-be73-cd121b4cea93 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.737355 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.737994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.738274 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.738549 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] PATH_INFO: `/v3/projects/885e3168528144da9db95492ee25be92/users/e8651f1c3b2043308e6777e3f4ebd7e7/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.753574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885e3168528144da9db95492ee25be92, user_id=e8651f1c3b2043308e6777e3f4ebd7e7, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.756519 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.769923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1337/2642] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:43 2022] PUT /identity/v3/projects/885e3168528144da9db95492ee25be92/users/e8651f1c3b2043308e6777e3f4ebd7e7/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:43.773803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.783547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb53c3-787f-45f8-bf2c-305bbe42befe admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.838136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a097-fa1b-4a6e-9f70-07ee94c31829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.838718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a097-fa1b-4a6e-9f70-07ee94c31829 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.838997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a097-fa1b-4a6e-9f70-07ee94c31829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.839303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a097-fa1b-4a6e-9f70-07ee94c31829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.839917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8a097-fa1b-4a6e-9f70-07ee94c31829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.841492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8a097-fa1b-4a6e-9f70-07ee94c31829 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.851033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1306/2643] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:43.855061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:43.865702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1afe68-0437-489e-8fe1-31d75c2811e0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:43.872003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:43.872664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.872945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.873218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] PATH_INFO: `/v3/projects/885e3168528144da9db95492ee25be92/users/e8651f1c3b2043308e6777e3f4ebd7e7/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.875116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=885e3168528144da9db95492ee25be92, user_id=e8651f1c3b2043308e6777e3f4ebd7e7, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:43.877783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:43.889292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1338/2644] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:43 2022] PUT /identity/v3/projects/885e3168528144da9db95492ee25be92/users/e8651f1c3b2043308e6777e3f4ebd7e7/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:43.899547 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:43.899835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:43.900139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:43.928291 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] MFA Rules not processed for user `e8651f1c3b2043308e6777e3f4ebd7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:43.930032 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:43.954023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:43.999262 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1307/2645] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:44.010319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:44.012020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:44.044747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:44.087985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043837cc-eb5b-4e27-9b06-b665f8482d4a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:44.092034 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:44.092618 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:44.092895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:44.093167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:44.101941 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:44.103401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:44.113101 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1339/2646] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:44.160548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:44.162083 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:44.175641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f867433-9b77-4ca1-8f03-bf62263bb961 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:44.179515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f23cb-1e40-41c0-ac06-531332be8a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:44.180142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f23cb-1e40-41c0-ac06-531332be8a07 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:44.180454 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f23cb-1e40-41c0-ac06-531332be8a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:44.180753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f23cb-1e40-41c0-ac06-531332be8a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:44.239639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f23cb-1e40-41c0-ac06-531332be8a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:44.241053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f23cb-1e40-41c0-ac06-531332be8a07 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:44.250962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1308/2647] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:47.521335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.582708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3566829-19df-4de5-a6d8-9c1b457441cb service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.587211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.587840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.588118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.588389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] PATH_INFO: `/v3/users/75f4fa33abf54bdd9749e75bdc3681a8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.589497 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f4fa33abf54bdd9749e75bdc3681a8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.591460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.636420 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f4fa33abf54bdd9749e75bdc3681a8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:47.644609 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f4fa33abf54bdd9749e75bdc3681a8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:47.656989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f4fa33abf54bdd9749e75bdc3681a8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:47.657624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f4fa33abf54bdd9749e75bdc3681a8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:47.658233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f4fa33abf54bdd9749e75bdc3681a8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:47.670241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1340/2648] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:47 2022] DELETE /identity/v3/users/75f4fa33abf54bdd9749e75bdc3681a8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:47.674932 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.675432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.675553 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.694689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:47.696590 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:47.753302 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1309/2649] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:47.764070 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.765486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:47.809259 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:47.823826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920e0f7-ccbf-4004-9880-03a741b8aa87 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.827423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.828019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.828293 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.828561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.837238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.838650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.847832 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1341/2650] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:48.103472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:48.104882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:48.116711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84b6a6-4bca-426d-85ba-dcf97eb1c3ff None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:48.122174 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:48.122803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.123120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.123426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] PATH_INFO: `/v3/projects/b71fd98477e04a55aebc7dc2b44c2610` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.124572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71fd98477e04a55aebc7dc2b44c2610)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:48.126515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:48.174696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71fd98477e04a55aebc7dc2b44c2610', 'request_id': 'req-6a1ed47c-5a83-42c5-9e9f-d82786145b71'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.178171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:48.181968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71fd98477e04a55aebc7dc2b44c2610', 'request_id': 'req-6a1ed47c-5a83-42c5-9e9f-d82786145b71'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.189447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71fd98477e04a55aebc7dc2b44c2610', 'request_id': 'req-6a1ed47c-5a83-42c5-9e9f-d82786145b71'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.200343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] The token cache is being invalidate because project b71fd98477e04a55aebc7dc2b44c2610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:48.200632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.201260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.202762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1310/2651] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:48 2022] DELETE /identity/v3/projects/b71fd98477e04a55aebc7dc2b44c2610 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:48.206889 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:48.216887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2867d-283c-457d-980d-94de2c29a315 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:48.272323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:48.272946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.273227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.273500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] PATH_INFO: `/v3/users/e8651f1c3b2043308e6777e3f4ebd7e7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.275103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8651f1c3b2043308e6777e3f4ebd7e7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:48.277045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:48.328983 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8651f1c3b2043308e6777e3f4ebd7e7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.337184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8651f1c3b2043308e6777e3f4ebd7e7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.349151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8651f1c3b2043308e6777e3f4ebd7e7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.349783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8651f1c3b2043308e6777e3f4ebd7e7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.350392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-59e8823c-3fb6-42d7-9ae4-472432df8566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8651f1c3b2043308e6777e3f4ebd7e7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.363577 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1342/2652] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:48 2022] DELETE /identity/v3/users/e8651f1c3b2043308e6777e3f4ebd7e7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:48.637074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:48.638683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:48.653990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ed47c-5a83-42c5-9e9f-d82786145b71 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:48.710218 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:48.710904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.711787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.712070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] PATH_INFO: `/v3/projects/885e3168528144da9db95492ee25be92` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.713179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885e3168528144da9db95492ee25be92)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:48.715080 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:48.759906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885e3168528144da9db95492ee25be92', 'request_id': 'req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.767483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885e3168528144da9db95492ee25be92', 'request_id': 'req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.773301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885e3168528144da9db95492ee25be92', 'request_id': 'req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.784649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] The token cache is being invalidate because project 885e3168528144da9db95492ee25be92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:48.784941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.785998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1cc1c7cc-340a-4fdc-b47a-3e8d1f5ec520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:48.787420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1311/2653] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:48 2022] DELETE /identity/v3/projects/885e3168528144da9db95492ee25be92 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:48.792472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.792943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.793035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.812513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:48.813701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:48.871895 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1343/2654] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:48.879837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.880126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.880402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.900694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:48.901897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:48.909739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1312/2655] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:48.913883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:48.918652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:48.925218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a2be7-db62-409a-943e-c8f2a6cf5606 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:48.931797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:48.935992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.935992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.935992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.935992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:48.935992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:48.935992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:48.949343 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1344/2656] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:48.953905 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:48.963242 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a9da2-2c1d-41dd-965f-7f07c6019f8e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:48.967903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:48.968494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:48.968772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:48.969041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:48.969670 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:48.971607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:48.998304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1313/2657] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:48 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:49.002448 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.013241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc66dc8-f679-41ce-b7bf-1210b4eeae4e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.068446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.069364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.069815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.070291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.071723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.073636 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.094926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1345/2658] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:49.098980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.109536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cab9ac-987b-4837-a96d-91d9428a19f1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.114086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.114684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.114965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.115265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.115935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.118010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.127656 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1314/2659] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:49.131563 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.142423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f09f0-abaf-4abd-bf0b-4fbd1cd97033 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.147019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e641b1-ff91-4e22-bc1a-ce715c05eda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.147653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e641b1-ff91-4e22-bc1a-ce715c05eda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.147928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e641b1-ff91-4e22-bc1a-ce715c05eda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.148196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e641b1-ff91-4e22-bc1a-ce715c05eda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.148857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e641b1-ff91-4e22-bc1a-ce715c05eda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.150453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e641b1-ff91-4e22-bc1a-ce715c05eda4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.160817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1346/2660] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:49.164891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.174999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6f892-77ee-48f5-96c7-43918fa2c855 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.179616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.180750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.181033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.181309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] PATH_INFO: `/v3/projects/f2b55b6ec2ec4819a67764c35b2a7e8d/users/9f8bbae3296643278e37fc39a1834d42/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.196153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b55b6ec2ec4819a67764c35b2a7e8d, user_id=9f8bbae3296643278e37fc39a1834d42, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.198799 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.212077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1315/2661] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:49 2022] PUT /identity/v3/projects/f2b55b6ec2ec4819a67764c35b2a7e8d/users/9f8bbae3296643278e37fc39a1834d42/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:49.224047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.224288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.224559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.254702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] MFA Rules not processed for user `9f8bbae3296643278e37fc39a1834d42`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:49.256009 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:49.324607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1347/2662] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:49.329198 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.338868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:49.390475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4a429-5794-47fa-902b-5c0bc49fa863 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.395043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.395660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.395935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.396203 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.396814 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.398708 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.423968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1316/2663] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:49.428126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.439724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467f556-e0ae-4d83-9d52-0d11133ecd85 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.493137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:49.499926 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.500542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.500985 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.501256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.501886 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.503928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.525656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1348/2664] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:49.529731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.540238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a18e7-5b48-4447-a802-6b2f1f829259 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.544870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.545456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.545733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.546001 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.546629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.548236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.560253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1317/2665] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:49.564191 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.573912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff6fa9-6f4d-4088-8608-c570268fb075 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.578472 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56114651-9b10-4dfe-9628-99124e67efd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.579056 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114651-9b10-4dfe-9628-99124e67efd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.579365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114651-9b10-4dfe-9628-99124e67efd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.579640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56114651-9b10-4dfe-9628-99124e67efd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.580266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114651-9b10-4dfe-9628-99124e67efd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.581840 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56114651-9b10-4dfe-9628-99124e67efd3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.591449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1349/2666] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:49.595171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.605266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabd0db-d50c-4a58-97f2-9817087102c6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.609860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.610485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.610762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.611033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] PATH_INFO: `/v3/projects/f530de5ad6b6426ca43fec243ddfd91d/users/9ee423190573488691a08cdef9a44fca/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.625911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f530de5ad6b6426ca43fec243ddfd91d, user_id=9ee423190573488691a08cdef9a44fca, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.628568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.641435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1318/2667] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:49 2022] PUT /identity/v3/projects/f530de5ad6b6426ca43fec243ddfd91d/users/9ee423190573488691a08cdef9a44fca/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:49.652439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.652724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.652996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.680565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] MFA Rules not processed for user `9ee423190573488691a08cdef9a44fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:49.681755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:49.742785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1350/2668] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:49.747594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.809524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2fa5-dd98-4855-9a88-f7558f7918fc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.814151 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.814748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.815027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.815346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.815983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.817868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.843895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1319/2669] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:49.848023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.859942 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442cdbe-1256-4d0f-90fd-45052b2d18cf None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.915639 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.916236 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.916510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.916777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.917401 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.919323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.939747 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1351/2670] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:49 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:49.943667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.953884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba316c79-7e98-48f9-8e5f-a58478eb0886 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.958526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.959127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.959436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.959706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:49.960335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:49.962392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:49.972776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1320/2671] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:49.977240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:49.987137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106af18-0e3a-495c-a14a-a0bbe3d08558 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:49.991750 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:49.992387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:49.992666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:49.992991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] PATH_INFO: `/v3/projects/9e2fad50a5e6473ea0495d779076d075/users/5c4626a16d3348dc98c400041a479dc4/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:50.007273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2fad50a5e6473ea0495d779076d075, user_id=5c4626a16d3348dc98c400041a479dc4, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:50.010174 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:50.022715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1352/2672] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:49 2022] PUT /identity/v3/projects/9e2fad50a5e6473ea0495d779076d075/users/5c4626a16d3348dc98c400041a479dc4/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:50.027161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:50.036891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68360aed-e60c-443b-aadd-b0320f9b5208 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:50.065619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:50.093035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e53e3-2764-4dfe-b1ae-b313b11c4df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:50.093630 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e53e3-2764-4dfe-b1ae-b313b11c4df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:50.093909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e53e3-2764-4dfe-b1ae-b313b11c4df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:50.094179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e53e3-2764-4dfe-b1ae-b313b11c4df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:50.094804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e53e3-2764-4dfe-b1ae-b313b11c4df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:50.096407 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e53e3-2764-4dfe-b1ae-b313b11c4df2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:50.105668 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1321/2673] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:50.109869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:50.120248 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa33af-ed85-4f5d-a753-a1c7af5cc0b6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:50.124896 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:50.125539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:50.125816 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:50.126085 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] PATH_INFO: `/v3/projects/9e2fad50a5e6473ea0495d779076d075/users/5c4626a16d3348dc98c400041a479dc4/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:50.128012 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2fad50a5e6473ea0495d779076d075, user_id=5c4626a16d3348dc98c400041a479dc4, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:50.130637 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:50.141470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1353/2674] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:50 2022] PUT /identity/v3/projects/9e2fad50a5e6473ea0495d779076d075/users/5c4626a16d3348dc98c400041a479dc4/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:50.153012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:50.153300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:50.153573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:50.180731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] MFA Rules not processed for user `5c4626a16d3348dc98c400041a479dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:50.181967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:50.249651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1322/2675] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:50.260820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:50.262517 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:50.282812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:50.325995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690de242-5864-40e2-b96c-4e96475da595 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:50.329698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:50.330303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:50.330584 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:50.330859 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:50.339814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:50.341247 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:50.350546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1354/2676] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:50.392548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:50.393925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:50.406073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b30b-8e0a-4f9c-b1d5-f6ddb2e6e055 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:50.409597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5fae3-3cb6-410d-8b34-b2c202654089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:50.410182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5fae3-3cb6-410d-8b34-b2c202654089 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:50.410456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5fae3-3cb6-410d-8b34-b2c202654089 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:50.410723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5fae3-3cb6-410d-8b34-b2c202654089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:50.467754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5fae3-3cb6-410d-8b34-b2c202654089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:50.469189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5fae3-3cb6-410d-8b34-b2c202654089 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:50.480053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1323/2677] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:53.353506 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.415093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792466d-9ce1-4574-8092-89cc8417794d service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.419558 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.420177 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.420515 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.420791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] PATH_INFO: `/v3/users/9f8bbae3296643278e37fc39a1834d42` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.421891 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8bbae3296643278e37fc39a1834d42)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.423837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.469168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8bbae3296643278e37fc39a1834d42'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.477972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8bbae3296643278e37fc39a1834d42'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.493284 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8bbae3296643278e37fc39a1834d42'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.493925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8bbae3296643278e37fc39a1834d42'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.494526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8bbae3296643278e37fc39a1834d42'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.506904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1355/2678] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:53 2022] DELETE /identity/v3/users/9f8bbae3296643278e37fc39a1834d42 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:53.511624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.512114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.512245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.531389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:53.532550 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.577293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:53.589830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1324/2679] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:53.599465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.600866 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.643787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:53.658841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6286e-923d-4096-bac3-222698595172 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.662529 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.663133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.663460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.663737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.673333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.674761 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.684176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1356/2680] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:53.950170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.951611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.964710 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef4791-b1f2-4be4-8b73-e05b83543819 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.969285 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.969938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.970213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.970483 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] PATH_INFO: `/v3/projects/f2b55b6ec2ec4819a67764c35b2a7e8d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.971619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b55b6ec2ec4819a67764c35b2a7e8d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.973541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.019330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b55b6ec2ec4819a67764c35b2a7e8d', 'request_id': 'req-1d438616-92fb-4a0a-9002-1ed57aba44f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.027130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b55b6ec2ec4819a67764c35b2a7e8d', 'request_id': 'req-1d438616-92fb-4a0a-9002-1ed57aba44f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.032824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b55b6ec2ec4819a67764c35b2a7e8d', 'request_id': 'req-1d438616-92fb-4a0a-9002-1ed57aba44f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.043676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] The token cache is being invalidate because project f2b55b6ec2ec4819a67764c35b2a7e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:54.043965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.044593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.046060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1325/2681] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:53 2022] DELETE /identity/v3/projects/f2b55b6ec2ec4819a67764c35b2a7e8d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:54.049997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:54.059711 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0eff85-e4fd-4330-9948-2e4318b75b85 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:54.116397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:54.116936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.117212 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.117482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] PATH_INFO: `/v3/users/9ee423190573488691a08cdef9a44fca` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.118580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee423190573488691a08cdef9a44fca)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:54.120540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.165407 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee423190573488691a08cdef9a44fca'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.172203 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee423190573488691a08cdef9a44fca'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.184543 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee423190573488691a08cdef9a44fca'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.185185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee423190573488691a08cdef9a44fca'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.185774 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee423190573488691a08cdef9a44fca'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.198325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1357/2682] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:54 2022] DELETE /identity/v3/users/9ee423190573488691a08cdef9a44fca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:54.479070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:54.480627 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:54.494425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d438616-92fb-4a0a-9002-1ed57aba44f5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:54.549546 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:54.550183 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.550458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.550730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] PATH_INFO: `/v3/projects/f530de5ad6b6426ca43fec243ddfd91d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.551865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f530de5ad6b6426ca43fec243ddfd91d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:54.553762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.599890 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f530de5ad6b6426ca43fec243ddfd91d', 'request_id': 'req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.607272 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f530de5ad6b6426ca43fec243ddfd91d', 'request_id': 'req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.612706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f530de5ad6b6426ca43fec243ddfd91d', 'request_id': 'req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.624750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] The token cache is being invalidate because project f530de5ad6b6426ca43fec243ddfd91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:54.625041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.625682 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.627163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1326/2683] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:54 2022] DELETE /identity/v3/projects/f530de5ad6b6426ca43fec243ddfd91d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:54.631090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:54.632499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:54.645416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba28050-ffb3-46a0-b17f-e55abfbee639 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:54.707351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:54.707987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.708270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.708559 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] PATH_INFO: `/v3/users/5c4626a16d3348dc98c400041a479dc4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.709669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4626a16d3348dc98c400041a479dc4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:54.711653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.713919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:54.878669 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4626a16d3348dc98c400041a479dc4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.887486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4626a16d3348dc98c400041a479dc4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.900156 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4626a16d3348dc98c400041a479dc4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.900821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4626a16d3348dc98c400041a479dc4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.901433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d1594655-c4e8-4bfc-af77-30eeccfb7347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4626a16d3348dc98c400041a479dc4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.913417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1358/2684] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:54 2022] DELETE /identity/v3/users/5c4626a16d3348dc98c400041a479dc4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:55.188629 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.190257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.204635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f53b-4ef2-4732-b8ae-f99182dbb6e8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.208386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:55.265206 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.265894 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.266177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.266447 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] PATH_INFO: `/v3/projects/9e2fad50a5e6473ea0495d779076d075` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.267618 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2fad50a5e6473ea0495d779076d075)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.269535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.314461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2fad50a5e6473ea0495d779076d075', 'request_id': 'req-062a3184-4d07-4603-99f4-872247dae086'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:55.321558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2fad50a5e6473ea0495d779076d075', 'request_id': 'req-062a3184-4d07-4603-99f4-872247dae086'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:55.327459 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2fad50a5e6473ea0495d779076d075', 'request_id': 'req-062a3184-4d07-4603-99f4-872247dae086'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:55.337999 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] The token cache is being invalidate because project 9e2fad50a5e6473ea0495d779076d075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:55.338284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:55.338898 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-062a3184-4d07-4603-99f4-872247dae086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:55.340405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1327/2685] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:55 2022] DELETE /identity/v3/projects/9e2fad50a5e6473ea0495d779076d075 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:55.346285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.346573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.346850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.366642 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:55.368465 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.425220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1359/2686] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:55.434154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.434436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.434706 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.453723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:55.454900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.462671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1328/2687] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:55.466831 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.477116 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a3d32-62d7-4d7c-bddd-d59f8c220558 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.481739 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.482340 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.482619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.482893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.483570 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:55.483877 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.485475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.497535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1360/2688] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:55.501971 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.511104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2873bc-4c2a-490e-8876-8b172b514c63 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.515726 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.516250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.516523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.516789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.517401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.519292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.546090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1329/2689] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:55.550143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.560065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c710ee-dafa-4472-b9c5-71ac54747a3e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.615242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.615839 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.616118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.616391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.617008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.618943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.635604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:55.646535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1361/2690] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:55.647640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.658364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90381b-14a1-452d-ab39-18b77ff8e838 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.662918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.663542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.663882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.664175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.664917 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.666502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.676345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1330/2691] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:55.680713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.690511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aec3f4-d5ea-4f81-8670-d7688701ca90 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.695140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f09f9-47c2-432f-8d3b-a79c0a6f140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.696361 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f09f9-47c2-432f-8d3b-a79c0a6f140b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.696640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f09f9-47c2-432f-8d3b-a79c0a6f140b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.696913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f09f9-47c2-432f-8d3b-a79c0a6f140b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.697546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f09f9-47c2-432f-8d3b-a79c0a6f140b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.699120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f09f9-47c2-432f-8d3b-a79c0a6f140b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.708822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1362/2692] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:55.712759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.722462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80954ae-d974-4c3f-a4d5-c0c517eb28ea admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.727082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.727747 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.728033 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.728306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] PATH_INFO: `/v3/projects/7b73f2a035c24791a4f24b596bd51962/users/fb8cfd81f76c4ccda2d366fc332c19de/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.742832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b73f2a035c24791a4f24b596bd51962, user_id=fb8cfd81f76c4ccda2d366fc332c19de, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.745529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.758951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1331/2693] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:55 2022] PUT /identity/v3/projects/7b73f2a035c24791a4f24b596bd51962/users/fb8cfd81f76c4ccda2d366fc332c19de/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:55.763956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa899c-770e-4ded-a9db-17e7e4be80df None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.764241 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa899c-770e-4ded-a9db-17e7e4be80df None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.764512 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa899c-770e-4ded-a9db-17e7e4be80df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.782900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-33aa899c-770e-4ded-a9db-17e7e4be80df None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:55.784049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-33aa899c-770e-4ded-a9db-17e7e4be80df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.841267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1363/2694] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:55.850760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.852465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.887588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:55.909541 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2dfe57-8d17-473d-a9d5-4509ad47b625 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.913156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.913753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.914089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.914366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.923179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.924624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.934415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1332/2695] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:56.785050 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248d780-a963-4445-8924-26e9028bce46 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:56.785583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248d780-a963-4445-8924-26e9028bce46 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:56.785583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248d780-a963-4445-8924-26e9028bce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:56.813740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7248d780-a963-4445-8924-26e9028bce46 None None] MFA Rules not processed for user `fb8cfd81f76c4ccda2d366fc332c19de`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:56.814955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7248d780-a963-4445-8924-26e9028bce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:56.881353 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1364/2696] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:56.886662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:56.896035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c090b9-fb65-4673-afc9-a66aac9b7cf8 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:56.901084 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:56.901671 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:56.901945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:56.902211 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:56.902829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:56.904743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:56.930554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1333/2697] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:56 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:56.934400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248d780-a963-4445-8924-26e9028bce46 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:56.945028 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248d780-a963-4445-8924-26e9028bce46 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:56.999922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:57.000530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:57.000807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:57.001078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:57.001706 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:57.003789 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:57.025226 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1365/2698] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:56 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:57.029193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:57.039276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30570d6f-fc86-45dd-b300-c009a1cd0fb1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:57.043743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:57.044327 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:57.044603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:57.044875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:57.045501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:57.047094 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:57.057250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1334/2699] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:57.061295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:57.071823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ed7d2-a2a4-4b40-99e9-3ec604a17ff1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:57.076380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:57.077022 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:57.077300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:57.077573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] PATH_INFO: `/v3/projects/ff18913e0e75421f9838242e9d8fa466/users/020d43903db34edeab6a6af97ec7b5c6/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:57.079787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:57.093900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff18913e0e75421f9838242e9d8fa466, user_id=020d43903db34edeab6a6af97ec7b5c6, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:57.096583 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:57.109014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1366/2700] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:57 2022] PUT /identity/v3/projects/ff18913e0e75421f9838242e9d8fa466/users/020d43903db34edeab6a6af97ec7b5c6/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:57.113027 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:57.124498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a1a2d-4cf6-47f3-bd3b-059f59dc4a85 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:57.181436 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758235a-1607-4902-b10a-50a9bae100f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:57.181960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758235a-1607-4902-b10a-50a9bae100f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:57.182255 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758235a-1607-4902-b10a-50a9bae100f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:57.182522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758235a-1607-4902-b10a-50a9bae100f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:57.183153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758235a-1607-4902-b10a-50a9bae100f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:57.184777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758235a-1607-4902-b10a-50a9bae100f0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:57.194139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1335/2701] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:57.198843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:57.208913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58267edb-4b1f-47e1-ad11-51e78376ec24 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:57.213534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:57.214175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:57.214456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:57.214728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] PATH_INFO: `/v3/projects/ff18913e0e75421f9838242e9d8fa466/users/020d43903db34edeab6a6af97ec7b5c6/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:57.216654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff18913e0e75421f9838242e9d8fa466, user_id=020d43903db34edeab6a6af97ec7b5c6, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:57.219324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:57.229376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1367/2702] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:57 2022] PUT /identity/v3/projects/ff18913e0e75421f9838242e9d8fa466/users/020d43903db34edeab6a6af97ec7b5c6/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:57.903082 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:57.903702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:57.903702 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:57.933503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] MFA Rules not processed for user `020d43903db34edeab6a6af97ec7b5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:57.934718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:58.004005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1336/2703] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:06:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:58.016564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:58.018425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:58.076586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51c3e-59e5-4547-9502-0910894f62e9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:58.080254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:58.080852 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:58.081126 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:58.081396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:58.137296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:58.138727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:58.147446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1368/2704] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:59.505232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:59.506633 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:59.518653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c8087-a21f-49bb-bfeb-0dfba86adefb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:59.522180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:59.522769 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:59.523051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:59.523870 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:59.532566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:59.533976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:59.544079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1337/2705] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:08.143470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:09.539129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:12.457227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:12.521509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb360c4-948e-4252-9421-9ba0b737f71e service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:12.525988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:12.526607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:12.526881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:12.527151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] PATH_INFO: `/v3/users/fb8cfd81f76c4ccda2d366fc332c19de` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:12.528311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8cfd81f76c4ccda2d366fc332c19de)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:12.530218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:12.574182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8cfd81f76c4ccda2d366fc332c19de'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.581610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8cfd81f76c4ccda2d366fc332c19de'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.593486 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8cfd81f76c4ccda2d366fc332c19de'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.594122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8cfd81f76c4ccda2d366fc332c19de'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.594728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8cfd81f76c4ccda2d366fc332c19de'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.607016 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1369/2706] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:12 2022] DELETE /identity/v3/users/fb8cfd81f76c4ccda2d366fc332c19de => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:12.775943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:12.786632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a00739-c214-4451-96d2-d7a30a58bae5 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:12.843176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:12.843843 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:12.844127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:12.844406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] PATH_INFO: `/v3/projects/7b73f2a035c24791a4f24b596bd51962` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:12.845528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b73f2a035c24791a4f24b596bd51962)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:12.847462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:12.895566 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b73f2a035c24791a4f24b596bd51962', 'request_id': 'req-cba7f024-a769-49fe-89e3-dd97f744c3b4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.903397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b73f2a035c24791a4f24b596bd51962', 'request_id': 'req-cba7f024-a769-49fe-89e3-dd97f744c3b4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.908939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b73f2a035c24791a4f24b596bd51962', 'request_id': 'req-cba7f024-a769-49fe-89e3-dd97f744c3b4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.924644 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] The token cache is being invalidate because project 7b73f2a035c24791a4f24b596bd51962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:12.924939 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.925591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:12.927061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1338/2707] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:12 2022] DELETE /identity/v3/projects/7b73f2a035c24791a4f24b596bd51962 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:12.931053 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:12.932565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:12.945427 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd05053-43ff-4142-8811-46d76d5d28aa admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.001095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.001651 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.001928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.002199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] PATH_INFO: `/v3/users/020d43903db34edeab6a6af97ec7b5c6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.003450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=020d43903db34edeab6a6af97ec7b5c6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.005368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.027038 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:13.051499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '020d43903db34edeab6a6af97ec7b5c6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.058657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '020d43903db34edeab6a6af97ec7b5c6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.070431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '020d43903db34edeab6a6af97ec7b5c6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.071060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '020d43903db34edeab6a6af97ec7b5c6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.071686 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68e97a6a-54d8-40b0-acab-4f1ebca3c76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '020d43903db34edeab6a6af97ec7b5c6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.083809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1370/2708] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:12 2022] DELETE /identity/v3/users/020d43903db34edeab6a6af97ec7b5c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:13.254417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.256096 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:13.271169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f024-a769-49fe-89e3-dd97f744c3b4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.332796 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.333423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.333704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.333978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] PATH_INFO: `/v3/projects/ff18913e0e75421f9838242e9d8fa466` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.335086 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff18913e0e75421f9838242e9d8fa466)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.337577 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.370130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:13.384966 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff18913e0e75421f9838242e9d8fa466', 'request_id': 'req-03703170-6b87-47a1-9bc4-437203e6b63e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.391401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff18913e0e75421f9838242e9d8fa466', 'request_id': 'req-03703170-6b87-47a1-9bc4-437203e6b63e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.396794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff18913e0e75421f9838242e9d8fa466', 'request_id': 'req-03703170-6b87-47a1-9bc4-437203e6b63e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.407975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] The token cache is being invalidate because project ff18913e0e75421f9838242e9d8fa466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:13.408257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.408875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-03703170-6b87-47a1-9bc4-437203e6b63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:13.410304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1339/2709] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:13 2022] DELETE /identity/v3/projects/ff18913e0e75421f9838242e9d8fa466 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:13.415303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.415596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.415910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.434801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:13.436032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:13.494790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1371/2710] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:13.502544 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.502829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.503106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.523415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:13.525109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:13.533172 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1340/2711] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:13.537459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.546935 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1796b3-d5f1-4c3f-95ae-b70af84c38a1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.551507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.552098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.552380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.552655 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.553296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:07:13.553601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.555250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.566685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1372/2712] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:07:13.571117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.579938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d543d3-3ed5-462a-b466-8b277f07d874 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.584508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.585081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.585355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.585621 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.586780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.588676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.614108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1341/2713] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:07:13 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:07:13.618078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.627606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df8be-3723-48ea-82f8-e1aba01bea7c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.683450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.684122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.684352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.684619 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.685242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.687141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.708207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1373/2714] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:07:13 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:07:13.712309 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.722392 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b31a-9179-4c55-ac3b-b413603c06e1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.727012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.727661 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.727942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.728216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.728855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.730434 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.739995 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1342/2715] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:07:13.744196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.753845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8b928-332f-466a-a93f-b009114a6810 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.759234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84468b2-4811-4a4e-b495-f98e98db2e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.759824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84468b2-4811-4a4e-b495-f98e98db2e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.760121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84468b2-4811-4a4e-b495-f98e98db2e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.760392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84468b2-4811-4a4e-b495-f98e98db2e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.761018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84468b2-4811-4a4e-b495-f98e98db2e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.762593 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84468b2-4811-4a4e-b495-f98e98db2e9e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.772837 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1374/2716] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:07:13.776868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.786801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ff6a0-d7c2-4e1e-8d2b-568fb3a695c9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:13.791445 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:13.792079 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.792363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.792653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] PATH_INFO: `/v3/projects/4fa52419b91642039f7023341235d92f/users/3bfd75811901498c83bda0f89768c955/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.807969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa52419b91642039f7023341235d92f, user_id=3bfd75811901498c83bda0f89768c955, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:13.810616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:13.823783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1343/2717] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:07:13 2022] PUT /identity/v3/projects/4fa52419b91642039f7023341235d92f/users/3bfd75811901498c83bda0f89768c955/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:13.835140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:13.835456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:13.835727 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:13.844704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:13.868854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] MFA Rules not processed for user `3bfd75811901498c83bda0f89768c955`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:13.870072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:13.932654 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1375/2718] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:13.943704 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:13.945406 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:14.000251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3941238-b8e3-4373-ae62-82ac26fa0101 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:14.003851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656f826-3428-409f-9385-ef71525be2f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '623f9046c3924b35b36d8807f8eef54c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:14.004439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656f826-3428-409f-9385-ef71525be2f5 service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:14.004718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656f826-3428-409f-9385-ef71525be2f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:14.004989 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656f826-3428-409f-9385-ef71525be2f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:14.013903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656f826-3428-409f-9385-ef71525be2f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:14.015347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656f826-3428-409f-9385-ef71525be2f5 service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:14.023888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1344/2719] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:07:20.876317 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:20.877751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:20.947587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec58fa-b1e6-47a0-b641-cae0db85749d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:20.953024 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:20.953624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:20.953898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:20.954163 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:20.955860 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:20.967530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:20.969081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:20.979644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1376/2720] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:24.020012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:30.974808 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=62 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:35.310548 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:35.311039 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:35.311161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:35.342574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:35.343841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:35.404153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1345/2721] 10.222.0.66 () {58 vars in 967 bytes} [Thu Sep 22 15:07:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:35.450616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:35.452065 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:35.519483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d3dd4-e97f-4b60-a320-017afbf5bf0e service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:35.524661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00040a08-7a36-4bee-aa67-82cedab640c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:35.525264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00040a08-7a36-4bee-aa67-82cedab640c4 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:35.525544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00040a08-7a36-4bee-aa67-82cedab640c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:35.525950 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00040a08-7a36-4bee-aa67-82cedab640c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:35.534359 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00040a08-7a36-4bee-aa67-82cedab640c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:35.536000 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00040a08-7a36-4bee-aa67-82cedab640c4 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:35.544216 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1377/2722] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:07:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:07:36.958230 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:37.025243 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f9b07-f197-4968-a3ef-0d1d51c6233d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:37.029786 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:37.030429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.031278 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.031554 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] PATH_INFO: `/v3/users/3bfd75811901498c83bda0f89768c955` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.032703 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bfd75811901498c83bda0f89768c955)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:37.034910 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:37.082956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bfd75811901498c83bda0f89768c955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.091060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bfd75811901498c83bda0f89768c955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.091645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bfd75811901498c83bda0f89768c955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.105556 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bfd75811901498c83bda0f89768c955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.113429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bfd75811901498c83bda0f89768c955'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.121641 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1346/2723] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:36 2022] DELETE /identity/v3/users/3bfd75811901498c83bda0f89768c955 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:37.126135 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.126640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.126756 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.145482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:37.146681 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:37.205220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1378/2724] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:37.214737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:37.216193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:37.227384 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:37.283505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a823c-d157-4afa-9f7e-fa2856a14cdc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:37.288152 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24843c26-1782-4830-b4c0-4c63c3e336f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:37.288824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24843c26-1782-4830-b4c0-4c63c3e336f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.289098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24843c26-1782-4830-b4c0-4c63c3e336f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.289366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24843c26-1782-4830-b4c0-4c63c3e336f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.300464 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24843c26-1782-4830-b4c0-4c63c3e336f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:37.301944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24843c26-1782-4830-b4c0-4c63c3e336f5 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:37.311333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1347/2725] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:37.600213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:37.601444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:37.616690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb9ae2-43a0-4426-9072-a735372285bb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:37.622867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:37.623962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.624598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.625072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] PATH_INFO: `/v3/projects/4fa52419b91642039f7023341235d92f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.627455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa52419b91642039f7023341235d92f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:37.630413 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:37.691836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa52419b91642039f7023341235d92f', 'request_id': 'req-0e9dbfce-485d-454e-bab1-136951817200'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.703314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa52419b91642039f7023341235d92f', 'request_id': 'req-0e9dbfce-485d-454e-bab1-136951817200'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.709673 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa52419b91642039f7023341235d92f', 'request_id': 'req-0e9dbfce-485d-454e-bab1-136951817200'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.723819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] The token cache is being invalidate because project 4fa52419b91642039f7023341235d92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:37.724587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.724759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0e9dbfce-485d-454e-bab1-136951817200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:37.726483 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1379/2726] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:37 2022] DELETE /identity/v3/projects/4fa52419b91642039f7023341235d92f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:37.731603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.732258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.732794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.754820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:37.756904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:37.838153 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1348/2727] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:37.846788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.847120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.847377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.867645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:37.868344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:37.879605 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1380/2728] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:37.883823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:37.897044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4107b2-ceb5-4ab8-9a55-51ae3b368614 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:37.902404 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:37.903031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.903348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.903622 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.904756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:07:37.905066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:37.907340 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:37.924755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1349/2729] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:07:37.926297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:37.936239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4ae61-15d2-4725-8bff-495f5946787d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:37.942585 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:37.943240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:37.943510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:37.943772 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:37.944426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:37.946338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:37.953223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:37.979239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1381/2730] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:07:37 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:07:37.983216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:37.993865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4eb1e7-cd0b-40d9-b2a5-96316debe642 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:38.057482 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:38.058185 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:38.058539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:38.059020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:38.059652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:38.062333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:38.089176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1350/2731] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:07:37 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:07:38.093310 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:38.104571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b062c-2448-4bcf-bac2-d7e0af1d76bf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:38.110017 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:38.110813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:38.111256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:38.111542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:38.112196 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:38.113830 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:38.128871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1382/2732] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:07:38.131069 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:38.146293 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea562e08-4c2b-45d3-b158-50642ed0c217 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:38.153579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9952e-8017-4e48-91cc-0b3cfcb71d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:38.154213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9952e-8017-4e48-91cc-0b3cfcb71d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:38.154493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9952e-8017-4e48-91cc-0b3cfcb71d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:38.154768 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9952e-8017-4e48-91cc-0b3cfcb71d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:38.155653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9952e-8017-4e48-91cc-0b3cfcb71d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:38.157304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9952e-8017-4e48-91cc-0b3cfcb71d4a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:38.168559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1351/2733] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:07:38.172770 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:38.182925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1491c8f-75d2-4334-99aa-d7cc5dabbc26 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:38.188015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:38.188661 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:38.188936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:38.189207 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] PATH_INFO: `/v3/projects/eb5677466f5d4032a4cf6f05e34c3443/users/bc2c77c59d8741258476dbdf6c735506/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:38.204683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5677466f5d4032a4cf6f05e34c3443, user_id=bc2c77c59d8741258476dbdf6c735506, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:38.207447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:38.224817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1383/2734] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:07:38 2022] PUT /identity/v3/projects/eb5677466f5d4032a4cf6f05e34c3443/users/bc2c77c59d8741258476dbdf6c735506/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:38.230443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89cbf5-e094-46e0-ab61-b6cda0509a41 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:38.230806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89cbf5-e094-46e0-ab61-b6cda0509a41 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:38.231109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89cbf5-e094-46e0-ab61-b6cda0509a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:38.252263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8c89cbf5-e094-46e0-ab61-b6cda0509a41 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:38.253507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8c89cbf5-e094-46e0-ab61-b6cda0509a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:38.267488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:38.320555 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1352/2735] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:07:38.327350 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:38.329553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:38.392250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15270d5-1a99-4bbe-bdb3-2b4de2811b86 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:38.395822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:38.396422 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:38.396698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:38.396969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:38.406590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:38.408060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:38.419035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1384/2736] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:44.281542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.282132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.282365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.321057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] MFA Rules not processed for user `bc2c77c59d8741258476dbdf6c735506`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:44.322561 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:44.432885 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1353/2737] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:44.435149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.449694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85a11b-0096-4f8f-80f0-bfe18a7e9eda service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.455900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.456919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.457429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.458715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.459701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.462301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:44.513434 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1385/2738] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:07:44 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:07:44.515825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.532425 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38a019-f73e-415f-b5db-fcf40ef0e4ed None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.603788 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.604735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.605041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.605450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.606179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.609182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:44.646446 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.648821 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:44.666374 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1354/2739] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:07:44 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:07:44.667667 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.676433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d1f41-59a3-4692-a1bb-5b90392e6564 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.683927 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.684869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.685148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.685415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] PATH_INFO: `/v3/users/945b92d7096c456884ff41bad41bb0da` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.686660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=945b92d7096c456884ff41bad41bb0da)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.688461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8062efa-6f3e-4c5b-9819-ea398740f178 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.689881 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:44.700180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.701790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.702159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.704879 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.705521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.707512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:44.724414 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1355/2740] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:07:44.728871 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.744009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05b79c-7895-412a-bcd8-4e8ce38722a6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.752155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.752857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.753402 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.753688 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] PATH_INFO: `/v3/projects/0755e1f2ba6f4899986c5a02f2b03688/users/68ccf0a5ada348ee9bb08cae9fb484d4/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.763161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945b92d7096c456884ff41bad41bb0da'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:44.770752 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0755e1f2ba6f4899986c5a02f2b03688, user_id=68ccf0a5ada348ee9bb08cae9fb484d4, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.772644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945b92d7096c456884ff41bad41bb0da'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:44.773619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:44.788318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1356/2741] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:07:44 2022] PUT /identity/v3/projects/0755e1f2ba6f4899986c5a02f2b03688/users/68ccf0a5ada348ee9bb08cae9fb484d4/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:44.791870 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945b92d7096c456884ff41bad41bb0da'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:44.792526 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.792667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945b92d7096c456884ff41bad41bb0da'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:44.793326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945b92d7096c456884ff41bad41bb0da'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:44.794949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:44.808730 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1386/2742] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:44 2022] DELETE /identity/v3/users/945b92d7096c456884ff41bad41bb0da => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:07:44.813098 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:44.885893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74184853-5a02-4e80-ad21-7741beb0604e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.951451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.952068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.952386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.952666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.953356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.955014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:44.966815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1357/2743] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:07:44.971705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:44.977809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:44.984324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224aec0-68a1-4db8-8957-536ebdbed530 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:44.991152 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:44.992189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:44.992660 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:44.993295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] PATH_INFO: `/v3/projects/0755e1f2ba6f4899986c5a02f2b03688/users/68ccf0a5ada348ee9bb08cae9fb484d4/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:44.995260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0755e1f2ba6f4899986c5a02f2b03688, user_id=68ccf0a5ada348ee9bb08cae9fb484d4, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:44.997999 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:45.012057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1387/2744] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:07:44 2022] PUT /identity/v3/projects/0755e1f2ba6f4899986c5a02f2b03688/users/68ccf0a5ada348ee9bb08cae9fb484d4/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:45.025980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:45.028301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:45.044602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667beb4-53f3-4075-88cb-1d562f17531f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:45.127835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:45.128802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:45.129375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:45.129940 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] PATH_INFO: `/v3/projects/7ade45dd41b848c5801b6d37a4232f86` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:45.131507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ade45dd41b848c5801b6d37a4232f86)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:45.135072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:45.196542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:45.198502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ade45dd41b848c5801b6d37a4232f86', 'request_id': 'req-747e9f31-f95a-4ac9-a163-299ecccb8423'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.224102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ade45dd41b848c5801b6d37a4232f86', 'request_id': 'req-747e9f31-f95a-4ac9-a163-299ecccb8423'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.235897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ade45dd41b848c5801b6d37a4232f86', 'request_id': 'req-747e9f31-f95a-4ac9-a163-299ecccb8423'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.265453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] The token cache is being invalidate because project 7ade45dd41b848c5801b6d37a4232f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:45.265754 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.267019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.268953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1358/2745] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:45 2022] DELETE /identity/v3/projects/7ade45dd41b848c5801b6d37a4232f86 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:45.272882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:45.275090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:45.291995 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49a265-57bd-4cd3-a76c-2e96d43ad452 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:45.381380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:45.382025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:45.382304 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:45.382576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] PATH_INFO: `/v3/users/188e5c33eff64b1c8893574ef358b107` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:45.383820 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=188e5c33eff64b1c8893574ef358b107)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:45.385979 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:45.451298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188e5c33eff64b1c8893574ef358b107'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.460910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188e5c33eff64b1c8893574ef358b107'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.491815 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188e5c33eff64b1c8893574ef358b107'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.493322 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188e5c33eff64b1c8893574ef358b107'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.494374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-35a98c6f-a655-4a84-aa9e-344a06002c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188e5c33eff64b1c8893574ef358b107'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.508507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1388/2746] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:45 2022] DELETE /identity/v3/users/188e5c33eff64b1c8893574ef358b107 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:45.706946 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:45.718532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e9f31-f95a-4ac9-a163-299ecccb8423 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:45.836221 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:45.838537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:45.839190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:45.839735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] PATH_INFO: `/v3/projects/6ae921f7a2cb48aaa698c627874b544e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:45.841155 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae921f7a2cb48aaa698c627874b544e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:45.845073 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:45.919161 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae921f7a2cb48aaa698c627874b544e', 'request_id': 'req-a82242d9-64e8-4236-a7d3-dbcfda41acc1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.934819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae921f7a2cb48aaa698c627874b544e', 'request_id': 'req-a82242d9-64e8-4236-a7d3-dbcfda41acc1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.943684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae921f7a2cb48aaa698c627874b544e', 'request_id': 'req-a82242d9-64e8-4236-a7d3-dbcfda41acc1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.959156 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] The token cache is being invalidate because project 6ae921f7a2cb48aaa698c627874b544e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:45.959502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.960263 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:45.963101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1359/2747] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:45 2022] DELETE /identity/v3/projects/6ae921f7a2cb48aaa698c627874b544e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:49.351172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db812d-2fcf-4378-a03e-40328df7e175 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:49.351689 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db812d-2fcf-4378-a03e-40328df7e175 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:49.351807 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db812d-2fcf-4378-a03e-40328df7e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:49.381744 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-89db812d-2fcf-4378-a03e-40328df7e175 None None] MFA Rules not processed for user `68ccf0a5ada348ee9bb08cae9fb484d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:49.383025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-89db812d-2fcf-4378-a03e-40328df7e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:49.445764 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:49.459573 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1389/2748] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:49.477123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:49.478270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:49.560385 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:49.597377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82242d9-64e8-4236-a7d3-dbcfda41acc1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:49.604018 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:49.604625 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:49.604903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:49.605176 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:49.606960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:49.666578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:49.668204 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:49.677875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1360/2749] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:50.731601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:50.732069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:50.732069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:50.758755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] MFA Rules not processed for user `bc2c77c59d8741258476dbdf6c735506`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:50.760678 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:50.771021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1390/2750] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:07:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:59.674077 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=35 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:00.752910 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:13.475552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:13.477012 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:13.536589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a049b-d450-401e-acb0-7049875d8523 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:13.540234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:13.540838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:13.541117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:13.541389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:13.543123 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:13.557564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:13.559058 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:13.570772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1361/2751] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:13.849014 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:13.850591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:13.909539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be447134-f9d1-4d47-a44b-4c8cebf438a3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:13.913067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:13.913610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:13.913888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:13.914158 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:13.915876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:13.925025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:13.926438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9aa9e0-66da-4f0d-b8b8-36ca82454c80 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:13.934213 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1391/2752] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:14.119061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:14.129274 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb00c-8258-4017-a234-9db7629530e1 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:14.132880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:14.133474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:14.133757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:14.134037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:14.136095 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:14.150421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:14.151882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:14.163619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1362/2753] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:16.705188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:16.705763 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:16.706222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:16.731674 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:08:16.733321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:16.808566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1392/2754] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:08:16.821257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:16.822887 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:16.898938 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801956c-adf3-40f8-837a-e51462fca948 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:16.904072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:16.904701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:16.904986 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:16.905287 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:16.916281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:16.917880 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:16.926771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1363/2755] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:08:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:08:23.433867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:23.444867 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f8f76-5ff1-4d70-83e2-d167a25ce274 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:23.448608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:23.449371 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:23.449709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:23.450046 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:23.452425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:23.576516 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:23.578086 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c285bb-7afb-4ce8-899c-5edebc75e607 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:23.600418 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1393/2756] 10.222.0.66 () {60 vars in 1396 bytes} [Thu Sep 22 15:08:23 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:26.923650 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=64 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:31.946686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:31.956791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d0af8-e74b-47e7-93ae-3bd1ddc3b99b service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:31.960382 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:31.960968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:31.961250 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:31.961525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:31.970522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:31.971981 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:31.984601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1364/2757] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:33.596049 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:36.053842 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:36.054404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:36.054404 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:36.102204 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:08:36.103435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:36.120526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1394/2758] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Sep 22 15:08:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:08:36.142325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:36.162698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70097ee-cf64-4d5c-8652-681c279c7219 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:36.167626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaab02-2510-447a-bd8d-df2e743f3539 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:36.168213 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaab02-2510-447a-bd8d-df2e743f3539 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:36.168491 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaab02-2510-447a-bd8d-df2e743f3539 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:36.168762 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaab02-2510-447a-bd8d-df2e743f3539 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:36.177104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaab02-2510-447a-bd8d-df2e743f3539 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:36.178655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaab02-2510-447a-bd8d-df2e743f3539 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:36.186878 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1365/2759] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:08:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:08:46.117569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=18 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:46.183534 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=16 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:54.025283 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:54.026613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:54.038601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfdf75-0026-443e-a8cc-82afa6d8c71d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:54.042192 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:54.042782 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:54.043060 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:54.043393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:54.045033 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:54.054129 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:54.055594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:54.063540 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1395/2760] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:56.076834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:56.077338 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:56.077451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:56.096569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] MFA Rules not processed for user `1e8df64cf5134a7e957de60145c8783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:08:56.097759 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:56.104725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1366/2761] 10.222.0.66 () {58 vars in 967 bytes} [Thu Sep 22 15:08:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:08:56.117443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:56.129195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43317c6-705e-4b68-8c78-50440df2a5cc service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:56.134061 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '543542d17db0495f8151e779e6c6218f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:56.134647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:56.134925 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:56.135478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:56.143370 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:56.144934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:56.152824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1396/2762] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Sep 22 15:08:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:09:06.090757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:09:06.149560 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=12 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:09:56.038551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:09:56.054683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe1cf-835c-45c6-b730-7ca5174c72e0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:09:56.068519 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:09:56.069355 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:09:56.069738 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:09:56.070056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:09:56.072016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:09:56.088154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:09:56.089962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:09:56.101730 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1367/2763] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:09:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:00.074923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:00.084663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed628-1dfe-48eb-b561-7617f4667142 service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:00.088331 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:00.088924 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:00.089201 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:00.089470 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:00.091195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:00.154282 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:00.155776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:00.165588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1397/2764] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:03.228348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:03.230562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:03.250691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b07ac5-c6d5-435b-acc7-9b71e8a3a7e2 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:03.258065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:03.258921 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.259439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.260763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] PATH_INFO: `/v3/users/2febff9d4cbb476c90037ffacf44465d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.262260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2febff9d4cbb476c90037ffacf44465d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:03.264387 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:03.318579 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2febff9d4cbb476c90037ffacf44465d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.327120 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2febff9d4cbb476c90037ffacf44465d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.327822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2febff9d4cbb476c90037ffacf44465d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.340117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2febff9d4cbb476c90037ffacf44465d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.348660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-84bb25a2-622a-4efe-a75d-4c330c6b3c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2febff9d4cbb476c90037ffacf44465d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.357133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1368/2765] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:10:03 2022] DELETE /identity/v3/users/2febff9d4cbb476c90037ffacf44465d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:03.535374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:03.546708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1a075-abfa-441e-ae91-89d3ac6ee3ec service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:03.605607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:03.606232 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.606509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.606780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] PATH_INFO: `/v3/projects/58581ef3e3844816923467523d5be072` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.607899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] RBAC: Authorizing `identity:delete_project(project_id=58581ef3e3844816923467523d5be072)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:03.609797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:03.656104 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58581ef3e3844816923467523d5be072', 'request_id': 'req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.663296 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58581ef3e3844816923467523d5be072', 'request_id': 'req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.669032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58581ef3e3844816923467523d5be072', 'request_id': 'req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.680246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] The token cache is being invalidate because project 58581ef3e3844816923467523d5be072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:10:03.680534 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.681161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c2a1c3b1-9e0e-4cfa-8ab5-d57aaa88b88e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:03.682829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1398/2766] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:10:03 2022] DELETE /identity/v3/projects/58581ef3e3844816923467523d5be072 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:03.687603 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.687896 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.688179 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.708188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:03.709388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:03.769043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1369/2767] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:10:03.777755 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.777978 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.778249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.798105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:03.799335 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:03.807041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1399/2768] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:03.811324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:03.821655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a77ff-68fb-4d70-a117-0ed11acf46c4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:03.826214 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:03.826807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.827093 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.827400 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.828042 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:10:03.828345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:03.829928 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:03.841146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1370/2769] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:10:03.845380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:03.854450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa13692-2d27-4cf2-9888-2a9fa4672794 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:03.858975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:03.859607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.859882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.860676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.861332 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:03.863239 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:03.878607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:03.904311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1400/2770] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:10:03 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:10:03.909016 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:03.918875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed6b9-704c-4ce0-a400-542e2d5f0ee2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:03.962311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:03.976760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:03.977358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:03.977634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:03.977904 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:03.978551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:03.980479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:04.000943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1371/2771] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:10:03 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:10:04.005157 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:04.015817 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac13cf7-0301-41c6-be46-38b3b0f8ca20 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:04.020902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:04.021498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:04.021777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:04.022047 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:04.022683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:04.024307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:04.034143 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1401/2772] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:10:04.037883 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:04.048177 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7110c1-629f-447b-a58b-d120bcf9c629 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:04.053212 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41959b0f-9353-47cd-a863-3a45857b54ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:04.053833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959b0f-9353-47cd-a863-3a45857b54ff admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:04.054115 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959b0f-9353-47cd-a863-3a45857b54ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:04.054389 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41959b0f-9353-47cd-a863-3a45857b54ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:04.055056 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41959b0f-9353-47cd-a863-3a45857b54ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:04.056680 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41959b0f-9353-47cd-a863-3a45857b54ff admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:04.067610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1372/2773] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:10:04.071624 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:04.081377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb924e-2f40-4d10-8de6-377206e8dc12 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:04.085964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:04.086600 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:04.086879 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:04.087153 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] PATH_INFO: `/v3/projects/97d3ac84a92a42b1adc0ffb829e0de6f/users/5e769df45a434f9bb6cae5c3655e45c8/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:04.102295 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d3ac84a92a42b1adc0ffb829e0de6f, user_id=5e769df45a434f9bb6cae5c3655e45c8, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:04.104997 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:04.119051 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1402/2774] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:10:04 2022] PUT /identity/v3/projects/97d3ac84a92a42b1adc0ffb829e0de6f/users/5e769df45a434f9bb6cae5c3655e45c8/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:04.124088 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17914e44-f0a6-4250-a0ae-8e74f831361d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:04.124376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17914e44-f0a6-4250-a0ae-8e74f831361d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:04.124687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17914e44-f0a6-4250-a0ae-8e74f831361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:04.143537 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-17914e44-f0a6-4250-a0ae-8e74f831361d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:04.144727 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-17914e44-f0a6-4250-a0ae-8e74f831361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:04.203167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1373/2775] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:04.212893 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:04.214572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:04.271846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6cbc-0794-4ea8-924e-b302f48d6257 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:04.275580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:04.276180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:04.276494 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:04.276768 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:04.286571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:04.287977 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:04.297285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1403/2776] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:08.205200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8823bb-3d21-423a-aadc-4778272051c0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:08.205707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8823bb-3d21-423a-aadc-4778272051c0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:08.205810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8823bb-3d21-423a-aadc-4778272051c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:08.234772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ff8823bb-3d21-423a-aadc-4778272051c0 None None] MFA Rules not processed for user `5e769df45a434f9bb6cae5c3655e45c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:08.236640 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ff8823bb-3d21-423a-aadc-4778272051c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:08.305982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1374/2777] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:08.320996 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:08.322507 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:08.397137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b059d65-7a6f-412a-a77b-9236c2df98f7 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:08.402501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:08.403186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:08.403492 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:08.403759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:08.411812 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:08.413478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:08.422249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1404/2778] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:09.704499 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:09.705037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:09.705037 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:09.723916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] MFA Rules not processed for user `5e769df45a434f9bb6cae5c3655e45c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:09.725125 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:09.731963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1375/2779] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:10.041708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:10.051803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955512-3428-4b6d-8d6d-1c6f03851b1e service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:10.055742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:10.056387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:10.056712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:10.056994 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:10.058917 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:10.069644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:10.071115 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:10.079518 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1405/2780] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:10.414171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:10.415803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:10.467822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:10.499783 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35d349-3ab2-47e2-8080-efcca5da8c39 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:10.505565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:10.506269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:10.506615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:10.506962 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:10.508801 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:10.520778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:10.522476 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:10.534776 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1376/2781] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:18.059790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.075509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9117da-aa2e-4533-ac9e-4d659f816841 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.081931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.082539 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.082818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.083103 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.084987 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:18.098118 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.099776 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.110406 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1406/2782] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:10:20.528531 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=10 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:22.230435 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:22.232224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:22.246791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac3cd3-322f-41f2-bb59-3cbb0a196729 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:22.252764 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:22.253356 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:22.253632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:22.253903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:22.255624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:22.268787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:22.270593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:22.281804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1377/2783] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:28.105721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:32.277109 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:11:03.043809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:11:03.045374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:11:03.057298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97458f4-2eb1-4018-9d5f-e658c4a1dbc6 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:11:03.060885 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:11:03.061479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:11:03.061805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:11:03.062638 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:11:03.064499 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:11:03.074453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:11:03.075921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:11:03.084164 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1407/2784] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:11:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:11:03.158953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:11:03.168349 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d591-7baa-4379-9a1a-6a69cae09535 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:11:03.171970 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:11:03.172570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:11:03.172829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:11:03.173113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:11:03.174848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:11:03.188117 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:11:03.189532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:11:03.200164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1378/2785] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:11:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:11:13.080574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:11:13.195438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:12:23.108729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:12:23.110136 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:12:23.124498 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2781d13-f36f-4eeb-a6c9-8db61e29d0a6 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:12:23.129336 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:12:23.129919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:12:23.130193 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:12:23.130459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:12:23.132137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:12:23.143428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:12:23.144986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:12:23.155140 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1408/2786] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:12:33.150490 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:12:54.558811 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:12:54.567615 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b801fb4-09ab-4a0d-a9b7-6e2e6b6b068c service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:12:54.571178 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:12:54.571867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:12:54.572148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:12:54.572422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:12:54.574146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:12:54.632133 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:12:54.633588 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:12:54.642570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1379/2787] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:12:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:01.720220 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:01.729604 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bf33-8514-47d3-b80c-9397cfa7921b service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:01.733653 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:01.734237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:01.734514 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:01.734783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:01.736511 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:01.748328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:01.749754 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:01.758949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1409/2788] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:04.557562 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:04.574604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c490-67f1-4818-b17b-0af0bac5d1e8 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:04.580256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:04.580895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:04.581173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:04.582060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:04.583514 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:04.602290 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:04.605365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:04.621847 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1380/2789] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:07.685120 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:07.697097 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcac01-0a4b-456a-806c-1da851335e25 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:07.701968 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:07.702586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:07.702863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:07.703128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:07.705892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:07.717829 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:07.720228 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:07.733928 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1410/2790] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:13:07.773154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:07.774770 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:07.788433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8fa08-2306-475b-9e4c-d3349c469413 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:07.798248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:07.798889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:07.799163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:07.799470 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] PATH_INFO: `/v3/users/bc2c77c59d8741258476dbdf6c735506` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:07.800600 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2c77c59d8741258476dbdf6c735506)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:07.802664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:07.867273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2c77c59d8741258476dbdf6c735506'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:07.877354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2c77c59d8741258476dbdf6c735506'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:07.877980 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2c77c59d8741258476dbdf6c735506'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:07.891822 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2c77c59d8741258476dbdf6c735506'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:07.899146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2c77c59d8741258476dbdf6c735506'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:07.905295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1381/2791] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:13:07 2022] DELETE /identity/v3/users/bc2c77c59d8741258476dbdf6c735506 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:08.087081 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:08.099337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e574dc6-d78a-4ebd-83d8-e5c4625cd435 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:08.161387 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:08.161993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.162273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.162546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] PATH_INFO: `/v3/projects/eb5677466f5d4032a4cf6f05e34c3443` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.163723 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5677466f5d4032a4cf6f05e34c3443)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:08.165667 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:08.220148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5677466f5d4032a4cf6f05e34c3443', 'request_id': 'req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.227576 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5677466f5d4032a4cf6f05e34c3443', 'request_id': 'req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.233183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5677466f5d4032a4cf6f05e34c3443', 'request_id': 'req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.244615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] The token cache is being invalidate because project eb5677466f5d4032a4cf6f05e34c3443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:13:08.244903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.245550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.247054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1411/2792] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:13:08 2022] DELETE /identity/v3/projects/eb5677466f5d4032a4cf6f05e34c3443 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:08.251279 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:08.252725 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:08.267310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743089e-0dc9-4242-9325-687932b20d45 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:08.322943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:08.331192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:08.332516 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.332881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.333652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] PATH_INFO: `/v3/users/68ccf0a5ada348ee9bb08cae9fb484d4` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.335108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ccf0a5ada348ee9bb08cae9fb484d4)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:08.337234 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:08.384200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ccf0a5ada348ee9bb08cae9fb484d4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.391907 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ccf0a5ada348ee9bb08cae9fb484d4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.392492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ccf0a5ada348ee9bb08cae9fb484d4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.404626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ccf0a5ada348ee9bb08cae9fb484d4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.411265 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5c0e2939-846f-4c64-8187-91110ae59705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ccf0a5ada348ee9bb08cae9fb484d4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.417182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1382/2793] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:13:08 2022] DELETE /identity/v3/users/68ccf0a5ada348ee9bb08cae9fb484d4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:08.588110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:08.589705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:08.604276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920badc1-e7d2-4fa3-9cd7-c6ff36b48666 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:08.660548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:08.661168 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.661444 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.661712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] PATH_INFO: `/v3/projects/0755e1f2ba6f4899986c5a02f2b03688` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.662818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0755e1f2ba6f4899986c5a02f2b03688)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:08.664742 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:08.670883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:08.711393 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0755e1f2ba6f4899986c5a02f2b03688', 'request_id': 'req-db520687-04ec-4aa3-bb06-3457d72e3eb6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.718640 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0755e1f2ba6f4899986c5a02f2b03688', 'request_id': 'req-db520687-04ec-4aa3-bb06-3457d72e3eb6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.724504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0755e1f2ba6f4899986c5a02f2b03688', 'request_id': 'req-db520687-04ec-4aa3-bb06-3457d72e3eb6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.735325 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] The token cache is being invalidate because project 0755e1f2ba6f4899986c5a02f2b03688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:13:08.735614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.736244 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-db520687-04ec-4aa3-bb06-3457d72e3eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:13:08.737712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1412/2794] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:13:08 2022] DELETE /identity/v3/projects/0755e1f2ba6f4899986c5a02f2b03688 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:08.742655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.742941 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.743277 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.763723 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:13:08.764922 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:08.821439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1383/2795] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:13:08.829819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.830110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.830388 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.849682 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:13:08.850888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:08.858708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1413/2796] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:13:08.862744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:08.872787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc127dc-c53b-4143-8633-07f748642f98 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:08.877405 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:08.877975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.878253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.878529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.879164 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:13:08.879528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:08.881105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:08.893044 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1384/2797] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:13:08.897190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:08.906258 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e4f5d-e4c3-4cf3-abf3-d35bb04bf695 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:08.910887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:08.911513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:08.911795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:08.912068 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:08.912718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:08.914614 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:08.943334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1414/2798] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:13:08 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:13:08.947492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:08.957803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d058ca-50d9-46da-bb4f-ae36a0707aa0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:09.012145 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:09.012690 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:09.013219 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:09.013492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:09.014106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:09.016040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:09.026303 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:09.041975 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1385/2799] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:13:08 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:13:09.046105 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:09.056405 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760ede-0c28-41df-be56-446e3105079d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:09.060967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:09.061553 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:09.061827 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:09.062096 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:09.062728 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:09.064321 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:09.075031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1415/2800] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:13:09.079053 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:09.089305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595d509-3892-4908-9f71-bb6c5b464f95 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:09.094750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eac8f8-c1a4-4afa-a608-b7228cf68377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:09.095388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac8f8-c1a4-4afa-a608-b7228cf68377 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:09.095672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac8f8-c1a4-4afa-a608-b7228cf68377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:09.095960 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eac8f8-c1a4-4afa-a608-b7228cf68377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:09.096616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eac8f8-c1a4-4afa-a608-b7228cf68377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:09.098191 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eac8f8-c1a4-4afa-a608-b7228cf68377 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:09.109023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1386/2801] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:13:09.113148 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:09.123676 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a90c5-5012-455e-97ae-7809b2c77836 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:09.128986 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:09.129629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:09.129903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:09.130170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] PATH_INFO: `/v3/projects/643f960acfbb4e5d8218e8dadc9066be/users/2aae391064c2473eb0233a03e8f7f29e/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:09.145298 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643f960acfbb4e5d8218e8dadc9066be, user_id=2aae391064c2473eb0233a03e8f7f29e, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:09.147946 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:09.162010 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1416/2802] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:13:09 2022] PUT /identity/v3/projects/643f960acfbb4e5d8218e8dadc9066be/users/2aae391064c2473eb0233a03e8f7f29e/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:09.167149 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21b321-432c-485d-b0af-e463665e4d84 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:09.167463 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21b321-432c-485d-b0af-e463665e4d84 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:09.167737 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21b321-432c-485d-b0af-e463665e4d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:09.185953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1a21b321-432c-485d-b0af-e463665e4d84 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:13:09.187142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1a21b321-432c-485d-b0af-e463665e4d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:09.249258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1387/2803] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:13:09.258964 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:09.260845 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:09.319717 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976fac3-40ed-4120-bb07-24b6ad1eb731 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:09.323300 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:09.323921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:09.324200 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:09.324469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:09.333250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:09.334665 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:09.344463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1417/2804] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:13.672124 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:13.672694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:13.672694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:13.701832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] MFA Rules not processed for user `2aae391064c2473eb0233a03e8f7f29e`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:13:13.703169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:13.772610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1388/2805] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:13:13.777841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:13.787760 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3777b-9314-45b1-a7df-1b81b294038b service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:13.792547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:13.793149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:13.793439 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:13.793712 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:13.794337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:13.796301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:13.824649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1418/2806] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:13:13.828859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:13.841180 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0793b-f2a4-46a4-83e3-c6e836279da7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:13.901646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:13.902365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:13.902604 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:13.902877 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:13.903631 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:13.905572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:13.926502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1389/2807] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:13:13 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:13:13.930433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:13.941708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e2e7d-6360-49b7-b575-c9bda3928fa4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:13.946277 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:13.946901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:13.947183 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:13.947485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:13.948121 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:13.950411 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:13.956280 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:13.962513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1419/2808] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:13:13.966159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:13.971378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:13.977294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6585ea5-79a3-49e6-a4e5-126358a9411d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:13.982298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:13.982948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:13.983256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:13.983607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] PATH_INFO: `/v3/projects/6e7e2ed1be5e4a02945547beb389a752/users/a2924bd1523a4c6496c83e99b9fd8739/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:13.998744 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7e2ed1be5e4a02945547beb389a752, user_id=a2924bd1523a4c6496c83e99b9fd8739, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:14.001341 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:14.014419 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1390/2809] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:13:13 2022] PUT /identity/v3/projects/6e7e2ed1be5e4a02945547beb389a752/users/a2924bd1523a4c6496c83e99b9fd8739/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:14.018414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:14.028316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb90a81-8836-4264-8d5d-8f05b4051894 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:14.087766 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:14.088378 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:14.088658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:14.088931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:14.089571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:14.091172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:14.100887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1420/2810] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:13:14.105142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:14.115420 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e2814-e6bd-4315-a1bf-24726d0aab27 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:14.120054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:14.121260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:14.121507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:14.121807 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] PATH_INFO: `/v3/projects/6e7e2ed1be5e4a02945547beb389a752/users/a2924bd1523a4c6496c83e99b9fd8739/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:14.123868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7e2ed1be5e4a02945547beb389a752, user_id=a2924bd1523a4c6496c83e99b9fd8739, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:14.126522 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:14.137722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1391/2811] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:13:14 2022] PUT /identity/v3/projects/6e7e2ed1be5e4a02945547beb389a752/users/a2924bd1523a4c6496c83e99b9fd8739/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:13:14.311380 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:14.314227 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:14.389791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638935b6-6f31-459c-9e5f-5fe8a4d85958 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:14.448037 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:14.449269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:14.449607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:14.460719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:14.463122 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:14.544814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:14.548785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:14.550980 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582363cb-38d2-4a61-ab87-0f522a7cb610 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:14.564580 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1421/2812] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:15.930371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:15.944669 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a4d12-7239-491a-b361-3837353f04ae admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:15.948507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:15.949229 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:15.949574 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:15.949943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:15.951832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:15.982777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:15.984535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:15.997129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1392/2813] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:18.543008 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6b73-4ac2-4920-bac1-4d4ddbc2ddc1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:18.544302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6b73-4ac2-4920-bac1-4d4ddbc2ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:18.547972 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6b73-4ac2-4920-bac1-4d4ddbc2ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:18.616634 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b55d6b73-4ac2-4920-bac1-4d4ddbc2ddc1 None None] MFA Rules not processed for user `a2924bd1523a4c6496c83e99b9fd8739`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:13:18.618527 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b55d6b73-4ac2-4920-bac1-4d4ddbc2ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:18.743025 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1422/2814] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:13:18.757608 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:18.759048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:18.779838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf34bf2-79e0-4686-b07a-5d1be0e224f2 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:18.791426 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:18.792052 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:18.792377 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:18.792654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:18.858846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:18.860513 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:18.870735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1393/2815] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:28.739385 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=37 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:28.865651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:41.647888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:41.648417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:41.648417 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:41.667264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] MFA Rules not processed for user `2aae391064c2473eb0233a03e8f7f29e`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:13:41.669003 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:41.675668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1423/2816] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:13:51.661982 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:13:57.478397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:57.488142 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a787a8-3ed5-436b-be52-aa85cfb1a328 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:57.491736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:57.492332 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:57.492610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:57.492881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:57.494672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:13:57.504952 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:57.506366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9ccab-a67b-453f-84ee-1b4d709090f6 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:57.515126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1394/2817] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:13:59.257620 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:13:59.268108 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a360d0-addb-40c3-89ea-291f9ccbdb24 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:13:59.271564 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:13:59.272150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:13:59.272431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:13:59.272695 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:13:59.282256 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:13:59.283724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:13:59.294218 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1424/2818] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:14:07.511187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:14:09.289528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:14:32.900215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:32.900742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:32.900742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:32.919993 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] MFA Rules not processed for user `5e769df45a434f9bb6cae5c3655e45c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:14:32.921168 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:32.974296 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1395/2819] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:14:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:14:42.967498 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=25 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:01.306215 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:01.319365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f34fa5-72c9-439d-b67c-bcecccdae3ed service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:01.324644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:01.325254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:01.325551 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:01.325819 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:01.327556 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:01.337510 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:01.339064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4606f-8a6c-4477-bc10-b451072910da service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:01.347694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1425/2820] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:11.343787 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:21.853985 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:21.880587 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f99c69-de7f-45e0-a857-866798caa59d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:21.885967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:21.886563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:21.886867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:21.887104 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:21.889724 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:21.915020 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:21.916848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:21.929114 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1396/2821] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:24.673235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:24.673694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:24.673788 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:24.693169 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] MFA Rules not processed for user `2aae391064c2473eb0233a03e8f7f29e`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:16:24.694364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:24.701170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1426/2822] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:16:25.032471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:25.041696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e2180b-9d81-4669-baae-a8321753445a service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:25.045936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:25.046525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:25.046806 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:25.047081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:25.048841 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:25.058192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:25.059648 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:25.067523 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1397/2823] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:26.518185 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:26.519751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:26.590888 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf7f66-330b-4522-ad78-f6154b05fd81 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:26.597269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:26.597874 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:26.598713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:26.598713 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:26.600758 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:26.616952 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:26.618565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:26.629707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1427/2824] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:33.346492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:33.364655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699586-e8df-4624-86c7-60e7cfc46922 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:33.368316 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:33.368944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:33.369223 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:33.369496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:33.371318 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:33.464716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:33.466158 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:33.485475 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1398/2825] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:36.624795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=35 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:36.983062 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:36.984501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:37.007757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fbb42-8e1f-470f-9938-5b4089380187 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.012240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.012878 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.013162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.013431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] PATH_INFO: `/v3/users/5e769df45a434f9bb6cae5c3655e45c8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.014801 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e769df45a434f9bb6cae5c3655e45c8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.016734 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.065478 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e769df45a434f9bb6cae5c3655e45c8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.073130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e769df45a434f9bb6cae5c3655e45c8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.085622 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e769df45a434f9bb6cae5c3655e45c8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.086312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e769df45a434f9bb6cae5c3655e45c8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.086919 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-41c4eb76-c5c2-432c-9535-a96f908fece4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e769df45a434f9bb6cae5c3655e45c8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.099590 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1428/2826] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:16:36 2022] DELETE /identity/v3/users/5e769df45a434f9bb6cae5c3655e45c8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:16:37.274415 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.286699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1f9f-f037-499c-8202-323a97e8c8b7 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.344474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.345106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.345388 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.345659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] PATH_INFO: `/v3/projects/97d3ac84a92a42b1adc0ffb829e0de6f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.346784 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d3ac84a92a42b1adc0ffb829e0de6f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.348711 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.350102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:37.394944 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d3ac84a92a42b1adc0ffb829e0de6f', 'request_id': 'req-9fb02558-d3db-4ca5-9346-9e6dea045795'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.402903 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d3ac84a92a42b1adc0ffb829e0de6f', 'request_id': 'req-9fb02558-d3db-4ca5-9346-9e6dea045795'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.409066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d3ac84a92a42b1adc0ffb829e0de6f', 'request_id': 'req-9fb02558-d3db-4ca5-9346-9e6dea045795'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.420167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] The token cache is being invalidate because project 97d3ac84a92a42b1adc0ffb829e0de6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:16:37.420451 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.421070 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9fb02558-d3db-4ca5-9346-9e6dea045795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:37.422503 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1399/2827] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:16:37 2022] DELETE /identity/v3/projects/97d3ac84a92a42b1adc0ffb829e0de6f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:16:37.427561 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.427846 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.428119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.484456 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:16:37.485644 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:37.543098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1429/2828] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:16:37.550761 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.551045 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.551348 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.570653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:16:37.571881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:37.580148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1400/2829] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:16:37.584237 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.594264 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf98984-d1c8-48a3-8619-1c074124e442 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.598783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.599415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.599694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.599966 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.600601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:16:37.600904 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.602485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.613668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1430/2830] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:16:37.617819 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.627654 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283b19e-d30f-4ea2-91ce-24761e8a698e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.632379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.632977 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.633253 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.633524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.634146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.636071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.662950 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1401/2831] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:16:37 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:16:37.667119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.677072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d37be2-b0f0-4436-8194-46a49700eb83 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.689691 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:37.734790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.735436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.735719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.735993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.736627 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.738535 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.758887 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1431/2832] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:16:37 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:16:37.763779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.773982 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ede5e-877b-4f92-b0ef-274747847fb6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.778501 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.779078 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.779383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.779651 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.780276 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.781846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.791310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1402/2833] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:16:37.794961 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.804902 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f3cea-5729-46cd-9dfb-82a711d7e683 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.809395 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473abe9-bc2c-4938-8134-1296d262e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.810468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473abe9-bc2c-4938-8134-1296d262e82a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.810743 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473abe9-bc2c-4938-8134-1296d262e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.811011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473abe9-bc2c-4938-8134-1296d262e82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.811697 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473abe9-bc2c-4938-8134-1296d262e82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.813266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473abe9-bc2c-4938-8134-1296d262e82a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.823715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1432/2834] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:16:37.828197 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.837751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fe182-f6fa-49a1-91e6-a2c489706f0e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:37.842330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:37.842949 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.843251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.843524 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] PATH_INFO: `/v3/projects/fc8105a809364e67b8695f3e18a1d5e5/users/508f2e1664bd4fbab3a287062bf4e6a7/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.857653 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8105a809364e67b8695f3e18a1d5e5, user_id=508f2e1664bd4fbab3a287062bf4e6a7, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:37.860334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:37.872829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1403/2835] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:16:37 2022] PUT /identity/v3/projects/fc8105a809364e67b8695f3e18a1d5e5/users/508f2e1664bd4fbab3a287062bf4e6a7/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:16:37.877719 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cae6d-06d6-435c-bf59-6b501e0d64aa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:37.878032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cae6d-06d6-435c-bf59-6b501e0d64aa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:37.878307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cae6d-06d6-435c-bf59-6b501e0d64aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:37.897481 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-882cae6d-06d6-435c-bf59-6b501e0d64aa None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:16:37.898675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-882cae6d-06d6-435c-bf59-6b501e0d64aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:37.956267 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1433/2836] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:16:37.966009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:37.967789 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:38.033085 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d83839-47c9-46dd-9796-4ff7dc41eb60 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:38.036612 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:38.037227 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:38.037507 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:38.037780 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:38.048756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:38.050184 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:38.059359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1404/2837] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:42.062167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ec3c7-8343-4cab-822c-375661a6b9a0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:42.062729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ec3c7-8343-4cab-822c-375661a6b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:42.062729 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ec3c7-8343-4cab-822c-375661a6b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:42.091389 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-021ec3c7-8343-4cab-822c-375661a6b9a0 None None] MFA Rules not processed for user `508f2e1664bd4fbab3a287062bf4e6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:16:42.092601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-021ec3c7-8343-4cab-822c-375661a6b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:42.158458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1434/2838] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:16:42.172672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:42.174113 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:42.199911 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:42.266686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b61db-7c1a-4825-a5c4-1ca0b0e27201 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:42.272635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:42.273257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:42.273543 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:42.273816 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:42.282718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:42.284492 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:42.295118 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1405/2839] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:43.977737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:43.978272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:43.978408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:43.998184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] MFA Rules not processed for user `508f2e1664bd4fbab3a287062bf4e6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:16:43.999443 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:44.006913 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1435/2840] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:16:44.159914 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:44.162462 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:44.186642 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50021a90-b2af-4443-86ae-6742fd06582f service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:44.192857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:44.193606 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:44.193954 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:44.194493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:44.196696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:44.210956 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:44.212160 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:44.229269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1406/2841] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:49.041707 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:49.056077 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2cd7e-e3f6-4eef-a152-4401d9aadc0b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:49.061628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:49.062319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:49.062608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:49.062880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:49.064632 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:49.077089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:49.078594 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:49.089621 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1436/2842] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:54.223539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=31 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:59.084936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:17:29.764106 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:17:29.765765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:17:29.851607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0d054-fd7b-49cf-a614-3d4a47bf71c8 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:17:29.857301 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:17:29.858137 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:17:29.858422 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:17:29.858697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:17:29.860457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:17:29.872359 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:17:29.873918 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60fc3a-606f-4681-82e5-a12d0b01ad3b service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:17:29.883634 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1407/2843] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:17:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:17:39.879410 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=41 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:18:16.721437 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:18:16.722898 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:16.798548 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c71089-08c5-4176-870a-4e518708e1b7 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:18:16.802067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:18:16.802662 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:18:16.802936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:18:16.803223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:18:16.804937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:16.836608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:18:16.838030 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:18:16.857748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1437/2844] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:18:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:18:25.826515 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:18:25.827024 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:18:25.827148 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:18:25.845472 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] MFA Rules not processed for user `508f2e1664bd4fbab3a287062bf4e6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:18:25.846662 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:25.853380 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1408/2845] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:18:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:18:26.472223 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:18:26.473679 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:26.510078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbdb16-de46-4f1d-8393-6821fa3dc60c service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:18:26.513737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:18:26.514337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:18:26.515130 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:18:26.515445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:18:26.517199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:26.531824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:18:26.533260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dce8f7-a0be-4489-8ac0-322a82e6d567 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:18:26.546976 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1438/2846] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:18:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:18:35.840256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:18:36.542018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=62 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:18:39.012065 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:18:39.013474 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:39.025558 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50c395-b982-4416-8bae-fb4819255364 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:18:39.029802 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:18:39.030393 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:18:39.030672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:18:39.030943 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:18:39.032676 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:39.090818 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:18:39.092282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:18:39.105748 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1409/2847] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:18:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:18:49.102294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=28 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:18:50.136238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:18:50.136813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:18:50.136813 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:18:50.159992 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] MFA Rules not processed for user `2aae391064c2473eb0233a03e8f7f29e`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:18:50.161334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:50.169071 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1439/2848] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:18:50.486241 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:18:50.495638 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a225d0e-3ebe-46a4-a15d-206d2c228d6b service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:18:50.499701 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:18:50.500294 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:18:50.500575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:18:50.500851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:18:50.502557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:18:50.512632 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:18:50.514051 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:18:50.522258 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1410/2849] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:19:00.154197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:19:00.518624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:19:05.036374 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:19:05.046314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c648-57d0-4f61-9554-12a1b17bac0b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:19:05.049793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:19:05.050542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:19:05.050657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:19:05.050875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:19:05.052647 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:19:05.062799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:19:05.064275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6b068-02d6-4071-ba07-28879bca6d57 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:19:05.072319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1440/2850] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:19:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:19:07.780343 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:19:07.788996 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e16d11-7ad5-49e0-873f-a73dddcea654 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:19:07.792675 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:19:07.793261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:19:07.793540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:19:07.793815 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:19:07.795573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:19:07.858794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:19:07.859716 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:19:07.869207 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1411/2851] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:19:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:19:15.068623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:19:17.864881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=30 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:20:08.674658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:20:08.675231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:20:08.675351 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:20:08.694184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] MFA Rules not processed for user `508f2e1664bd4fbab3a287062bf4e6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:20:08.695402 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:20:08.702036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1441/2852] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:20:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:20:18.689396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:21:26.491867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:26.504236 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15c949-6cbe-4321-8900-0b7f1daa92c0 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:26.509833 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:26.510442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:26.510722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:26.510990 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:26.513333 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:26.526061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:26.527663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:26.539162 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1412/2853] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:21:29.344276 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:29.354344 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f097bb-e362-4b28-8e39-b7d3d596060a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:29.358617 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:29.359289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:29.359520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:29.359793 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:29.361509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:29.373067 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:29.374476 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:29.383790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1442/2854] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:21:32.199915 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:32.201536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:32.252101 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a247e9-3876-49c6-9773-804390a89355 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:32.257528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:32.258532 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:32.258821 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:32.259099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:32.260906 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:32.281757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:32.284489 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:32.299691 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1413/2855] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:21:35.488194 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:35.493154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:35.509144 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0641a5-a3bc-434c-9ad3-99e672d653e4 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:35.513828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:35.514449 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:35.514724 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:35.515301 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] PATH_INFO: `/v3/users/2aae391064c2473eb0233a03e8f7f29e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:35.516415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aae391064c2473eb0233a03e8f7f29e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:35.518326 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:35.564323 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aae391064c2473eb0233a03e8f7f29e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.571285 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aae391064c2473eb0233a03e8f7f29e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.582795 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aae391064c2473eb0233a03e8f7f29e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.583453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aae391064c2473eb0233a03e8f7f29e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.584045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aae391064c2473eb0233a03e8f7f29e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.595680 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1443/2856] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:21:35 2022] DELETE /identity/v3/users/2aae391064c2473eb0233a03e8f7f29e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:21:35.762559 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:35.773047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106ca3-1b03-4aab-a072-49e5e48282ba service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:35.828173 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:35.828804 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:35.829081 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:35.829351 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] PATH_INFO: `/v3/projects/643f960acfbb4e5d8218e8dadc9066be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:35.830457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643f960acfbb4e5d8218e8dadc9066be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:35.832383 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:35.880449 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643f960acfbb4e5d8218e8dadc9066be', 'request_id': 'req-541d336f-bde8-478e-b79c-c103243f64f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.887593 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643f960acfbb4e5d8218e8dadc9066be', 'request_id': 'req-541d336f-bde8-478e-b79c-c103243f64f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.894043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643f960acfbb4e5d8218e8dadc9066be', 'request_id': 'req-541d336f-bde8-478e-b79c-c103243f64f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.905418 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] The token cache is being invalidate because project 643f960acfbb4e5d8218e8dadc9066be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:21:35.905694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.906457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:35.907958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1414/2857] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:21:35 2022] DELETE /identity/v3/projects/643f960acfbb4e5d8218e8dadc9066be => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:21:35.911955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:35.913341 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:35.927305 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2d67e-b270-4a1b-a8f8-e98e89b6522b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:35.982503 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:35.983128 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:35.983445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:35.983721 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] PATH_INFO: `/v3/users/a2924bd1523a4c6496c83e99b9fd8739` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:35.984871 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2924bd1523a4c6496c83e99b9fd8739)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:35.986786 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:36.032528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2924bd1523a4c6496c83e99b9fd8739'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.039687 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2924bd1523a4c6496c83e99b9fd8739'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.045530 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:21:36.055023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2924bd1523a4c6496c83e99b9fd8739'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.055702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2924bd1523a4c6496c83e99b9fd8739'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.056288 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2924bd1523a4c6496c83e99b9fd8739'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.068260 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1444/2858] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:21:35 2022] DELETE /identity/v3/users/a2924bd1523a4c6496c83e99b9fd8739 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:21:36.237750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:36.239224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:36.254275 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d336f-bde8-478e-b79c-c103243f64f2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:36.310855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:36.311518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:36.311794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:36.312063 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] PATH_INFO: `/v3/projects/6e7e2ed1be5e4a02945547beb389a752` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:36.313216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7e2ed1be5e4a02945547beb389a752)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:36.315112 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:36.321175 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:21:36.363429 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7e2ed1be5e4a02945547beb389a752', 'request_id': 'req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.370567 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7e2ed1be5e4a02945547beb389a752', 'request_id': 'req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.376119 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7e2ed1be5e4a02945547beb389a752', 'request_id': 'req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.386772 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] The token cache is being invalidate because project 6e7e2ed1be5e4a02945547beb389a752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:21:36.387580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.388201 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8d4bc143-5a44-48cd-8ffd-ca22c39c1f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:21:36.389646 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1415/2859] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:21:36 2022] DELETE /identity/v3/projects/6e7e2ed1be5e4a02945547beb389a752 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:21:42.606504 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:42.608066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:42.663909 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2dc9a-f9b3-4a5f-9e14-2ec6518eec35 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:42.710075 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:42.710607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:42.710883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:42.711154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:42.712858 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:42.768250 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:42.769672 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:42.778272 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1445/2860] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:21:46.383884 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:21:52.774311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=77,keystone:DELETE=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:21:55.276143 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:55.276635 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:55.276750 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:55.296696 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] MFA Rules not processed for user `508f2e1664bd4fbab3a287062bf4e6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:21:55.297908 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:55.304663 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1416/2861] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:21:55.603352 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:55.604759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:55.659939 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96391338-0564-4580-9a98-65d7051bc3e0 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:55.663461 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:55.664054 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:55.664334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:55.664608 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:55.666338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:55.676043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:55.677467 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:55.685329 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1446/2862] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:21:56.259023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:56.261108 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:56.332071 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd308-a853-43d7-84e0-01b9cc82a52e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:56.337281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:56.337891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:56.338171 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:56.338442 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:56.340254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:56.351773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:56.353345 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:56.363129 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1417/2863] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:22:03.685176 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:22:03.686666 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:22:03.761809 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbfc91-d230-4a6e-a922-8cad43996afb service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:22:03.767569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:22:03.768170 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:22:03.768447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:22:03.768718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:22:03.770425 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:22:03.782623 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:22:03.784234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:22:03.794488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1447/2864] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:22:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:22:06.358830 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=35 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:22:06.658557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:22:06.672353 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491c85d-4890-4fb9-8e8b-4fd08b7fdb5d service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:22:06.677736 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:22:06.678334 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:22:06.678617 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:22:06.678888 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:22:06.680745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:22:06.693983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:22:06.695607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:22:06.706570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1418/2865] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:22:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:22:13.790119 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=60 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:22:16.701699 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:22:37.037477 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:22:37.048652 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80993f03-0a55-4ee2-8406-a2b6f80711f4 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:22:37.053525 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:22:37.054113 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:22:37.054390 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:22:37.054658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:22:37.056372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:22:37.067949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:22:37.069546 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:22:37.079905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1448/2866] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:22:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:22:47.074989 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:20.662182 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:20.663658 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:20.675371 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579e621-36ef-41a5-b736-9015df68e7ef service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:20.678878 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:20.679497 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:20.679779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:20.680048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:20.681695 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:20.695190 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:20.696626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:20.707310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1419/2867] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:29.998703 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:30.000089 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:30.011975 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c254632-c741-466a-aa01-f9dcfe9d8803 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:30.015675 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:30.016253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:30.016524 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:30.016785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:30.018424 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:30.078235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:30.079693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:30.088841 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1449/2868] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:30.702529 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:33.105450 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.116817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7300fce-a114-483a-98af-75fdf5fd2dcb service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.121755 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.122346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.122626 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.122900 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.124687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:33.136395 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.138002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.151417 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1420/2869] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:33.162230 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.164780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:33.185828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ce010-4592-42aa-9bdf-59e9a05e8ebd service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.191581 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.192235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.192505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.192954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] PATH_INFO: `/v3/users/508f2e1664bd4fbab3a287062bf4e6a7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.194240 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508f2e1664bd4fbab3a287062bf4e6a7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.196199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.248133 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508f2e1664bd4fbab3a287062bf4e6a7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.255412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508f2e1664bd4fbab3a287062bf4e6a7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.268125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508f2e1664bd4fbab3a287062bf4e6a7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.268777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508f2e1664bd4fbab3a287062bf4e6a7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.269383 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-464896a0-afd6-453c-9d26-3533cf9840c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508f2e1664bd4fbab3a287062bf4e6a7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.281415 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1450/2870] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:33 2022] DELETE /identity/v3/users/508f2e1664bd4fbab3a287062bf4e6a7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:33.465858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.487260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee583a6-8485-448a-a5ad-fe7a9f957de7 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.546616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.547607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.547882 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.548270 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] PATH_INFO: `/v3/projects/fc8105a809364e67b8695f3e18a1d5e5` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.549968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc8105a809364e67b8695f3e18a1d5e5)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.551951 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.596689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc8105a809364e67b8695f3e18a1d5e5', 'request_id': 'req-fe434840-70df-4732-a316-7ed4cbcd4aff'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.603974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc8105a809364e67b8695f3e18a1d5e5', 'request_id': 'req-fe434840-70df-4732-a316-7ed4cbcd4aff'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.609411 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc8105a809364e67b8695f3e18a1d5e5', 'request_id': 'req-fe434840-70df-4732-a316-7ed4cbcd4aff'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.620528 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] The token cache is being invalidate because project fc8105a809364e67b8695f3e18a1d5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:33.620812 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.621421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fe434840-70df-4732-a316-7ed4cbcd4aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:33.622852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1421/2871] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:33 2022] DELETE /identity/v3/projects/fc8105a809364e67b8695f3e18a1d5e5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:33.628468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.628749 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.629021 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.648238 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:33.649426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:33.706526 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1451/2872] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:33.714240 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.714520 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.714790 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.733929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:33.735126 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:33.743645 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1422/2873] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:33.747683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.757814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befabd70-2d58-489d-ba1d-ab86dedddcdb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.762334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.762933 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.763249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.763528 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.764249 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:23:33.764550 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.766396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.768019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:33.780663 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1452/2874] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:23:33.785216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.794299 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b310a63-5e9f-4c6d-a63b-2bd7540ce266 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.799867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.800461 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.800742 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.801127 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.801773 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.803729 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.839336 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1423/2875] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:33 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:33.843932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.854589 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78deb0b-615c-4397-8a68-a42f1c5645a9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.909602 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.910195 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.910473 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.910740 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.911403 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.913303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.934416 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1453/2876] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:33 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:23:33.938664 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.948961 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804f5f-6552-4a13-ab01-a5072b848ba9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.953624 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.954443 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.954728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.955002 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.955808 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.957659 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:33.970128 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1424/2877] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:33.974683 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:33.984536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294e6c9-d805-4b70-b324-dcbc0bb0ed7c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:33.989709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359410bc-0747-453d-8369-723337c12c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:33.990292 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359410bc-0747-453d-8369-723337c12c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:33.990587 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359410bc-0747-453d-8369-723337c12c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:33.990836 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359410bc-0747-453d-8369-723337c12c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:33.991495 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359410bc-0747-453d-8369-723337c12c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:33.993074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359410bc-0747-453d-8369-723337c12c6e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:34.003307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1454/2878] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:34.007257 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:34.017292 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3c1a-9461-4450-87bc-795e4cbf9bb6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:34.021824 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:34.022458 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.022739 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.023014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] PATH_INFO: `/v3/projects/d3e40f61fd2f42d5a82e88fd55493ee2/users/7f533be59862454c86546723fd8de2ef/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.038047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e40f61fd2f42d5a82e88fd55493ee2, user_id=7f533be59862454c86546723fd8de2ef, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:34.041092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:34.054140 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1425/2879] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:34 2022] PUT /identity/v3/projects/d3e40f61fd2f42d5a82e88fd55493ee2/users/7f533be59862454c86546723fd8de2ef/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:34.064922 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.065209 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.065482 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.092337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] MFA Rules not processed for user `7f533be59862454c86546723fd8de2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:34.093569 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:34.158479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1455/2880] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:34.170486 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:34.172193 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:34.178684 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:34.246311 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79add9ab-4ac9-4f33-b9c8-64bade4e8007 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:34.251852 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff1a6a40166e4f51a14cca9cd510e680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:34.252481 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.252760 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.253031 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.260733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:34.262307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:34.271757 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1426/2881] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:23:34.350181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:34.351616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:34.406684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec5d1b-c409-463b-829b-65a23e3bde7c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:34.410929 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:34.411598 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.411880 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.412151 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.413875 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:34.426937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:34.428396 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be6404-3074-46e2-9574-706a1d8e72b0 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:34.439048 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1456/2882] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:34.572969 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:34.633298 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7fdb-15ad-435d-abeb-a539d1feb42a service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:34.637838 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:34.638453 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.638733 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.639005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] PATH_INFO: `/v3/users/7f533be59862454c86546723fd8de2ef` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.640397 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f533be59862454c86546723fd8de2ef)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:34.642927 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:34.686439 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f533be59862454c86546723fd8de2ef'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:34.693423 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f533be59862454c86546723fd8de2ef'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:34.694010 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f533be59862454c86546723fd8de2ef'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:34.706254 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f533be59862454c86546723fd8de2ef'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:34.712983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f533be59862454c86546723fd8de2ef'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:34.718655 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1427/2883] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:34 2022] DELETE /identity/v3/users/7f533be59862454c86546723fd8de2ef => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:34.723137 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.723458 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.723736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.731571 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:34.742851 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:34.744074 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:34.800704 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1457/2884] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:34.810431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:34.811863 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:34.823043 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:34.870187 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0924-6726-4b57-aea6-0e6d39cb6c4b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:34.873860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340e3c2-265c-4f6a-9ec0-4c91a4d4f473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:34.874457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340e3c2-265c-4f6a-9ec0-4c91a4d4f473 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:34.874734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340e3c2-265c-4f6a-9ec0-4c91a4d4f473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:34.875005 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340e3c2-265c-4f6a-9ec0-4c91a4d4f473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:34.884398 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340e3c2-265c-4f6a-9ec0-4c91a4d4f473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:34.885817 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340e3c2-265c-4f6a-9ec0-4c91a4d4f473 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:34.896549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1428/2885] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:35.167962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.169364 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:35.182309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de015887-4eb1-499a-a9ff-f8333a9ea4c0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.186864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.187591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.187865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.188132 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] PATH_INFO: `/v3/projects/d3e40f61fd2f42d5a82e88fd55493ee2` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.189838 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e40f61fd2f42d5a82e88fd55493ee2)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.191775 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.239901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e40f61fd2f42d5a82e88fd55493ee2', 'request_id': 'req-6d285397-90fb-4bd1-b222-a1fda9f56475'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:35.246969 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e40f61fd2f42d5a82e88fd55493ee2', 'request_id': 'req-6d285397-90fb-4bd1-b222-a1fda9f56475'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:35.253099 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e40f61fd2f42d5a82e88fd55493ee2', 'request_id': 'req-6d285397-90fb-4bd1-b222-a1fda9f56475'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:35.263733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] The token cache is being invalidate because project d3e40f61fd2f42d5a82e88fd55493ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:35.264015 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:35.264629 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6d285397-90fb-4bd1-b222-a1fda9f56475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:35.266368 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1458/2886] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:35 2022] DELETE /identity/v3/projects/d3e40f61fd2f42d5a82e88fd55493ee2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:35.271378 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.271964 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.272072 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.291856 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:35.293064 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:35.351837 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1429/2887] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:35.359328 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.359805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.359948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.379731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:35.380932 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:35.388714 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1459/2888] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:35.393074 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.402861 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384e5c1-c688-445d-8854-e41ea111cc8a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.407530 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.408611 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.408892 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.409192 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.409832 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:23:35.410136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.411749 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.423421 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1430/2889] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:23:35.427963 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.437613 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687b599-9404-4660-8a1d-aca0a9959a1d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.447488 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.448092 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.448365 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.448628 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.449246 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.451138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.478923 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1460/2890] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:35.483306 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.494013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db691c2-4106-42c3-933c-2b12bdcf4fa4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.549875 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.550469 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.550746 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.551019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.551666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.553570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.574715 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1431/2891] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:23:35.578784 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.588956 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930df5f-97d6-43bf-b3e6-6228f351073c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.593429 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.594064 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.594338 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.594607 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.595752 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.597330 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.606943 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1461/2892] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:35.610897 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.621295 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25164200-62af-476c-b1a7-6645bf034dce admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.625936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe6a97-d325-42f8-82c8-c0ffcfc07c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.626575 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe6a97-d325-42f8-82c8-c0ffcfc07c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.626858 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe6a97-d325-42f8-82c8-c0ffcfc07c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.627135 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe6a97-d325-42f8-82c8-c0ffcfc07c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.627805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe6a97-d325-42f8-82c8-c0ffcfc07c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.629390 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe6a97-d325-42f8-82c8-c0ffcfc07c09 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.641030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1432/2893] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:35.645459 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.655269 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e94f51-fa8f-4f77-a70d-c1af9ac86d7e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.660397 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.661032 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.661307 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.661574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] PATH_INFO: `/v3/projects/bd0b022443eb493396b5947dbdf8399b/users/49a83826ab9c4c5aa7cb294347064473/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.675915 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0b022443eb493396b5947dbdf8399b, user_id=49a83826ab9c4c5aa7cb294347064473, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.678566 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.690818 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1462/2894] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:35 2022] PUT /identity/v3/projects/bd0b022443eb493396b5947dbdf8399b/users/49a83826ab9c4c5aa7cb294347064473/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:35.700734 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.701023 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.701326 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.729652 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] MFA Rules not processed for user `49a83826ab9c4c5aa7cb294347064473`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:35.730895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:35.735988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:35.796011 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1433/2895] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:35.800650 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.826531 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:35.863436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dac35-be27-479c-a5e9-7fbefec573bb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.868018 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.868615 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.868894 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.869166 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.869779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.871722 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:35.902748 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1463/2896] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:35.906889 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:35.918130 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4554d-b465-4f87-8990-494586ac3b00 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:35.975189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:35.975823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:35.976102 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:35.976535 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:35.977014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:35.978931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.000006 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1434/2897] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:35 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:36.004036 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.014146 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f449afc-2006-4b4b-b853-ccc6932a8f63 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.019505 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.020095 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.020372 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.020641 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.021270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.022857 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.033803 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1464/2898] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:36.037686 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.047942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84922e7-806b-4f04-946d-baa9bfae811e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.052565 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.053200 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.053480 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.053751 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] PATH_INFO: `/v3/projects/dd234ffecfb14f5080a2b1cb73f3d9f7/users/4c0b0d6d0a2e4fe08c33cf28f5376c3a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.068570 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd234ffecfb14f5080a2b1cb73f3d9f7, user_id=4c0b0d6d0a2e4fe08c33cf28f5376c3a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.071224 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.084433 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1435/2899] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:36 2022] PUT /identity/v3/projects/dd234ffecfb14f5080a2b1cb73f3d9f7/users/4c0b0d6d0a2e4fe08c33cf28f5376c3a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:23:36.088182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.097892 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61cebb-0cd7-4cf6-b96b-f9187dd73fc2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.154041 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d49e3-f7c3-4c82-85a3-fbbf1c42ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.154635 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d49e3-f7c3-4c82-85a3-fbbf1c42ab80 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.154908 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d49e3-f7c3-4c82-85a3-fbbf1c42ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.155173 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d49e3-f7c3-4c82-85a3-fbbf1c42ab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.155828 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d49e3-f7c3-4c82-85a3-fbbf1c42ab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.157408 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d49e3-f7c3-4c82-85a3-fbbf1c42ab80 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.166810 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1465/2900] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:36.170953 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.181687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcd528-30f4-4419-9f5b-12c2ab091feb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.186563 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.187215 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.187502 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.187771 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] PATH_INFO: `/v3/projects/dd234ffecfb14f5080a2b1cb73f3d9f7/users/4c0b0d6d0a2e4fe08c33cf28f5376c3a/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.190169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd234ffecfb14f5080a2b1cb73f3d9f7, user_id=4c0b0d6d0a2e4fe08c33cf28f5376c3a, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.192965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.202967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1436/2901] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:36 2022] PUT /identity/v3/projects/dd234ffecfb14f5080a2b1cb73f3d9f7/users/4c0b0d6d0a2e4fe08c33cf28f5376c3a/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:36.215098 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.215454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.215733 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.244804 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] MFA Rules not processed for user `4c0b0d6d0a2e4fe08c33cf28f5376c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:36.246029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:36.318186 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1466/2902] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:23:36.328779 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.331218 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:36.386572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb55b2-96d3-4e93-b19c-020cca38012a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.390386 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e074a0-0a3d-4c9c-afbc-d66ee31ca694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.390968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e074a0-0a3d-4c9c-afbc-d66ee31ca694 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.391266 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e074a0-0a3d-4c9c-afbc-d66ee31ca694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.391536 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e074a0-0a3d-4c9c-afbc-d66ee31ca694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.401054 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e074a0-0a3d-4c9c-afbc-d66ee31ca694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.402465 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e074a0-0a3d-4c9c-afbc-d66ee31ca694 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.411549 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1437/2903] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:36 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:36.558006 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.578901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:36.620574 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7be0e-d6ef-492c-87e9-12647d3cc1b7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.625045 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.625668 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.625948 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.626222 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] PATH_INFO: `/v3/users/49a83826ab9c4c5aa7cb294347064473` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.627955 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a83826ab9c4c5aa7cb294347064473)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.629876 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.673150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a83826ab9c4c5aa7cb294347064473'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:36.680921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a83826ab9c4c5aa7cb294347064473'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:36.693696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a83826ab9c4c5aa7cb294347064473'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:36.694347 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a83826ab9c4c5aa7cb294347064473'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:36.694954 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a83826ab9c4c5aa7cb294347064473'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:36.707102 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1467/2904] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:36 2022] DELETE /identity/v3/users/49a83826ab9c4c5aa7cb294347064473 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:36.711572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.711866 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.712146 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.719105 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:36.732041 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:36.733246 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:36.791019 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1438/2905] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:36.801199 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:36.802591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:36.858287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eb2e4-e7c6-43ad-b76b-f9650683e7d3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:36.862428 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:36.863019 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:36.863324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:36.863601 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:36.872252 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:36.873701 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:36.882833 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1468/2906] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:37.155111 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:37.156557 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:37.170916 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189b1a9-c9e4-4ca4-ba8c-4e1aedcd0659 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:37.175607 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:37.176735 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:37.177188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:37.177457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] PATH_INFO: `/v3/projects/bd0b022443eb493396b5947dbdf8399b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:37.178583 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0b022443eb493396b5947dbdf8399b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:37.180598 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:37.229493 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0b022443eb493396b5947dbdf8399b', 'request_id': 'req-bd8708c9-515a-497c-ad3f-14e82afe7e5b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.237191 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0b022443eb493396b5947dbdf8399b', 'request_id': 'req-bd8708c9-515a-497c-ad3f-14e82afe7e5b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.242805 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0b022443eb493396b5947dbdf8399b', 'request_id': 'req-bd8708c9-515a-497c-ad3f-14e82afe7e5b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.253810 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] The token cache is being invalidate because project bd0b022443eb493396b5947dbdf8399b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:37.254154 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.254825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.256335 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1439/2907] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:37 2022] DELETE /identity/v3/projects/bd0b022443eb493396b5947dbdf8399b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:37.260826 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:37.271375 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbcf86-b9c2-4aad-b69f-4ebcac665a2b service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:37.313299 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:37.331921 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:37.332544 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:37.332823 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:37.333093 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] PATH_INFO: `/v3/users/4c0b0d6d0a2e4fe08c33cf28f5376c3a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:37.334190 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0b0d6d0a2e4fe08c33cf28f5376c3a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:37.336149 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:37.383253 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0b0d6d0a2e4fe08c33cf28f5376c3a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.390436 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0b0d6d0a2e4fe08c33cf28f5376c3a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.402314 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0b0d6d0a2e4fe08c33cf28f5376c3a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.402945 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0b0d6d0a2e4fe08c33cf28f5376c3a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.403568 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8f3e610a-46c4-4b10-badb-582712ed8420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0b0d6d0a2e4fe08c33cf28f5376c3a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.415454 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1469/2908] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:37 2022] DELETE /identity/v3/users/4c0b0d6d0a2e4fe08c33cf28f5376c3a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:37.683282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:37.684820 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:37.699366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8708c9-515a-497c-ad3f-14e82afe7e5b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:37.756512 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:37.757139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:37.757438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:37.757713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] PATH_INFO: `/v3/projects/dd234ffecfb14f5080a2b1cb73f3d9f7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:37.758834 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd234ffecfb14f5080a2b1cb73f3d9f7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:37.760851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:37.791509 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:37.807782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd234ffecfb14f5080a2b1cb73f3d9f7', 'request_id': 'req-62d452c2-fa6f-4dd6-a630-23015389092f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.814542 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd234ffecfb14f5080a2b1cb73f3d9f7', 'request_id': 'req-62d452c2-fa6f-4dd6-a630-23015389092f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.819935 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd234ffecfb14f5080a2b1cb73f3d9f7', 'request_id': 'req-62d452c2-fa6f-4dd6-a630-23015389092f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.830494 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] The token cache is being invalidate because project dd234ffecfb14f5080a2b1cb73f3d9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:37.830782 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.831431 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-62d452c2-fa6f-4dd6-a630-23015389092f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:37.832893 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1440/2909] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:37 2022] DELETE /identity/v3/projects/dd234ffecfb14f5080a2b1cb73f3d9f7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:37.838035 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:37.838545 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:37.838991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:37.858616 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:37.859834 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:37.917610 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1470/2910] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:37.925403 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:37.925687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:37.925963 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:37.946068 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:37.947307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:37.955610 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1441/2911] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:37.959698 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:37.969433 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651730c-d3a0-4721-89e5-fc93e36e1c85 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:37.974565 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:37.975150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:37.975468 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:37.975759 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:37.976392 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:23:37.976690 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:37.978265 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:37.989799 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1471/2912] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:23:37.994060 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.003163 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f4976-083b-46e5-a64d-3d9b2def83d8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.007895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.008432 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.008713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.008988 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.009619 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.011569 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.039649 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1442/2913] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:37 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:38.043702 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.053586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84540ed-d8a7-4bf7-87b9-02e46872bfe3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.097500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:38.110400 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.111023 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.111333 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.111606 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.112229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.114125 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.134358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1472/2914] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:38 2022] POST /identity/v3/users => generated 278 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:23:38.138825 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.149965 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549e47-6e18-42d8-98e1-e8ce72535f37 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.154743 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.155437 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.155722 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.155997 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.156660 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.158791 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.169000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1443/2915] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:38.172805 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.182572 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdf8e-ffd2-4730-9159-75382f8c7e34 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.187042 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee090ff6-cd63-48ca-b031-4b58a6ab2c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.187659 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090ff6-cd63-48ca-b031-4b58a6ab2c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.187937 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090ff6-cd63-48ca-b031-4b58a6ab2c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.188205 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090ff6-cd63-48ca-b031-4b58a6ab2c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.188903 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee090ff6-cd63-48ca-b031-4b58a6ab2c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.190469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee090ff6-cd63-48ca-b031-4b58a6ab2c76 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.200738 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1473/2916] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:38.205225 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.215057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4333d-1afe-4035-820b-84fe79066ca4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.219763 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.221714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.221714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.221714 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] PATH_INFO: `/v3/projects/74894a0d2dd84280b2ba4f74f4046387/users/75fee4bfb2cf497491c03d7ca93e327a/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.235925 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74894a0d2dd84280b2ba4f74f4046387, user_id=75fee4bfb2cf497491c03d7ca93e327a, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.238591 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.251092 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1444/2917] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:38 2022] PUT /identity/v3/projects/74894a0d2dd84280b2ba4f74f4046387/users/75fee4bfb2cf497491c03d7ca93e327a/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:38.262485 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.262696 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.262936 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.295043 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] MFA Rules not processed for user `75fee4bfb2cf497491c03d7ca93e327a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:38.296270 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:38.358736 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1474/2918] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:38.363261 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.422745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a6dc-98d3-4035-92ce-575ad4bdfff8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.427379 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.427967 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.428239 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.428504 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.429121 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.430998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.457342 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1445/2919] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:38 2022] POST /identity/v3/projects => generated 357 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:38.461358 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.472235 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489222-925a-42f0-846e-22dbaca95ab9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.527783 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.528312 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.528586 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.528855 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.529474 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.531391 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.551779 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1475/2920] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:38 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:23:38.555698 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.566260 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989775b-cae2-4cd9-95c2-5ac476890955 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.570929 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.571551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.571829 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.572100 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.572728 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.574331 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.589245 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1446/2921] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:23:38.593658 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.603751 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc3ec94-eab2-47f2-8340-c18ed943ca0b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.608513 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.609162 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.609445 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.609718 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] PATH_INFO: `/v3/projects/a7baf945ed1c43b583a9ef3cc93aa587/users/3e99edf32f9b4f1c862d86cf3dfe86b6/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.624268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7baf945ed1c43b583a9ef3cc93aa587, user_id=3e99edf32f9b4f1c862d86cf3dfe86b6, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.628004 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.642993 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1476/2922] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:38 2022] PUT /identity/v3/projects/a7baf945ed1c43b583a9ef3cc93aa587/users/3e99edf32f9b4f1c862d86cf3dfe86b6/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:38.646955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.656860 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21db4e0-b745-4818-9703-cff888da11e9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.680167 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:38.714189 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ecc9f-cabd-4005-884c-cfa597b06216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.714846 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ecc9f-cabd-4005-884c-cfa597b06216 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.715057 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ecc9f-cabd-4005-884c-cfa597b06216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.715363 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ecc9f-cabd-4005-884c-cfa597b06216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.715994 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ecc9f-cabd-4005-884c-cfa597b06216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.717572 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ecc9f-cabd-4005-884c-cfa597b06216 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.727141 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1447/2923] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:38.731020 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.741685 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576488f-c436-417e-b7e7-31b5fe66bba9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.746167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.746797 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.747072 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.747377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] PATH_INFO: `/v3/projects/a7baf945ed1c43b583a9ef3cc93aa587/users/3e99edf32f9b4f1c862d86cf3dfe86b6/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.749254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7baf945ed1c43b583a9ef3cc93aa587, user_id=3e99edf32f9b4f1c862d86cf3dfe86b6, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.752509 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.762311 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1477/2924] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:38 2022] PUT /identity/v3/projects/a7baf945ed1c43b583a9ef3cc93aa587/users/3e99edf32f9b4f1c862d86cf3dfe86b6/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:38.772248 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.772602 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.772881 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.800256 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] MFA Rules not processed for user `3e99edf32f9b4f1c862d86cf3dfe86b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:38.801457 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:38.867793 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1448/2925] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:38.879588 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:38.881266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:38.892027 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:38.940078 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adae3e-289c-4742-9d79-0d5193c0614f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:38.943814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:38.944414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:38.944693 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:38.944962 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:38.954934 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:38.956412 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:38.965765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1478/2926] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:39.210188 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:39.219745 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365decf7-39d5-43fb-a325-9a4c827af6e9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:39.223718 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361e0bb-664a-4a84-ad49-9bff9cefab49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:39.224305 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361e0bb-664a-4a84-ad49-9bff9cefab49 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:39.224580 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361e0bb-664a-4a84-ad49-9bff9cefab49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:39.224848 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361e0bb-664a-4a84-ad49-9bff9cefab49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:39.279589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361e0bb-664a-4a84-ad49-9bff9cefab49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:39.281099 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361e0bb-664a-4a84-ad49-9bff9cefab49 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:39.290823 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1449/2927] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:48.961596 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=22 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:49.286700 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=79 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:54.090843 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:54.154419 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c1c07-ed43-4cfd-88bd-9101c70c7f4d service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:54.159536 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:54.160159 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:54.160438 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:54.160709 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] PATH_INFO: `/v3/users/75fee4bfb2cf497491c03d7ca93e327a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:54.161822 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fee4bfb2cf497491c03d7ca93e327a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:54.163780 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:54.210965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fee4bfb2cf497491c03d7ca93e327a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.218850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fee4bfb2cf497491c03d7ca93e327a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.234334 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fee4bfb2cf497491c03d7ca93e327a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.235970 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fee4bfb2cf497491c03d7ca93e327a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.236835 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fee4bfb2cf497491c03d7ca93e327a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.255309 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1479/2928] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:54 2022] DELETE /identity/v3/users/75fee4bfb2cf497491c03d7ca93e327a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:54.260324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:54.260862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:54.260862 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:54.280707 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:54.281923 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:54.344310 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1450/2929] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:54.354426 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:54.355850 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:54.415656 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505811b4-8f23-4a88-9140-7b0711fa9a6c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:54.419297 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:54.419900 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:54.420180 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:54.420453 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:54.429242 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:54.430664 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:54.440029 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1480/2930] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:54.699958 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:54.701525 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:54.717687 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481e3c5-b242-4edc-9c10-1a139a59021b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:54.723375 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:54.724007 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:54.724284 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:54.724666 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] PATH_INFO: `/v3/projects/74894a0d2dd84280b2ba4f74f4046387` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:54.725694 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74894a0d2dd84280b2ba4f74f4046387)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:54.727592 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:54.777767 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74894a0d2dd84280b2ba4f74f4046387', 'request_id': 'req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.787089 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74894a0d2dd84280b2ba4f74f4046387', 'request_id': 'req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.793368 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74894a0d2dd84280b2ba4f74f4046387', 'request_id': 'req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.805578 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] The token cache is being invalidate because project 74894a0d2dd84280b2ba4f74f4046387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:54.805948 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.806689 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.808488 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1451/2931] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:54 2022] DELETE /identity/v3/projects/74894a0d2dd84280b2ba4f74f4046387 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:23:54.812324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:54.824645 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52a558-a32b-4873-9a61-bcb6cbb3f47a service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:54.846414 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:54.887731 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:54.888268 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:54.888547 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:54.888824 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] PATH_INFO: `/v3/users/3e99edf32f9b4f1c862d86cf3dfe86b6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:54.890189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e99edf32f9b4f1c862d86cf3dfe86b6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:54.893090 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:54.939705 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e99edf32f9b4f1c862d86cf3dfe86b6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.948791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e99edf32f9b4f1c862d86cf3dfe86b6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.962150 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e99edf32f9b4f1c862d86cf3dfe86b6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.962912 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e99edf32f9b4f1c862d86cf3dfe86b6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.963649 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-88f67545-bc98-496f-a25b-5844dca3788a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e99edf32f9b4f1c862d86cf3dfe86b6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:54.976455 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1481/2932] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:54 2022] DELETE /identity/v3/users/3e99edf32f9b4f1c862d86cf3dfe86b6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:55.241753 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.243365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:55.259855 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2b578-e34d-4ae0-a32d-767fe3c4f13f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.318320 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.319061 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.319216 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.319539 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] PATH_INFO: `/v3/projects/a7baf945ed1c43b583a9ef3cc93aa587` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.320665 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7baf945ed1c43b583a9ef3cc93aa587)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.322589 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.352347 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:55.370300 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7baf945ed1c43b583a9ef3cc93aa587', 'request_id': 'req-78815033-7702-4c11-8860-bd19c9a6837b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:55.377683 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7baf945ed1c43b583a9ef3cc93aa587', 'request_id': 'req-78815033-7702-4c11-8860-bd19c9a6837b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:55.383408 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7baf945ed1c43b583a9ef3cc93aa587', 'request_id': 'req-78815033-7702-4c11-8860-bd19c9a6837b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:55.394330 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] The token cache is being invalidate because project a7baf945ed1c43b583a9ef3cc93aa587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:55.394616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:55.395269 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-78815033-7702-4c11-8860-bd19c9a6837b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:55.396731 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1452/2933] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:55 2022] DELETE /identity/v3/projects/a7baf945ed1c43b583a9ef3cc93aa587 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:55.402777 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.403273 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.403377 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.423110 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:55.424287 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:55.483475 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1482/2934] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:55.489685 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.489974 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.490249 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.510030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:55.511251 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:55.519354 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1453/2935] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:55.523466 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.533694 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec589c58-0516-437b-b43e-69b0dd60bd01 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.538172 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.538757 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.539058 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.539357 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.539990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:23:55.540289 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.541899 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.553765 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1483/2936] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Sep 22 15:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:23:55.557835 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.566945 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238156b-d69a-43e3-8e80-1e9bd137f5a6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.572568 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.573438 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.573872 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.574346 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.574983 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.576931 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.604597 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1454/2937] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:55 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:55.608710 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.619082 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe14a2-981e-404f-8de1-3fcbd3db15bc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.644275 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:55.676382 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.676905 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.677181 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.677447 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.678057 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.680013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.700167 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1484/2938] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:55 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:55.704307 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.714573 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a43e6-8c28-48cb-9dfe-d5b07b4a79aa admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.719170 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.720304 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.720581 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.720849 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.721479 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.723066 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.732713 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1455/2939] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:55.737088 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.746901 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4eb11-7135-4a19-bf5c-7d7fdf94d843 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.751657 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59a7ad-6873-487c-9f5b-7a566f8c9cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.752243 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59a7ad-6873-487c-9f5b-7a566f8c9cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.752521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59a7ad-6873-487c-9f5b-7a566f8c9cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.752790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59a7ad-6873-487c-9f5b-7a566f8c9cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.753450 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59a7ad-6873-487c-9f5b-7a566f8c9cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.755117 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59a7ad-6873-487c-9f5b-7a566f8c9cbf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.765684 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1485/2940] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:55.769639 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.779976 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2af04c-9b29-44c6-a86e-69fe6a490a4f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.785014 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.785672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.785968 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.786238 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] PATH_INFO: `/v3/projects/4c25e2747f134a32979baf911beef64f/users/f3a60df3212b4e4b869aad12eda4bb66/roles/074c5bafaca74f698537cbd8c86e21e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.801325 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c25e2747f134a32979baf911beef64f, user_id=f3a60df3212b4e4b869aad12eda4bb66, role_id=074c5bafaca74f698537cbd8c86e21e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.804013 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:55.816564 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1456/2941] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:55 2022] PUT /identity/v3/projects/4c25e2747f134a32979baf911beef64f/users/f3a60df3212b4e4b869aad12eda4bb66/roles/074c5bafaca74f698537cbd8c86e21e3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:23:55.827708 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.827990 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.828261 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.855175 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] MFA Rules not processed for user `f3a60df3212b4e4b869aad12eda4bb66`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:55.856520 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:55.919302 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1486/2942] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:55.923800 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:55.983756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30289c3b-5559-4e08-a264-d064f2238a2f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:55.988273 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:55.988857 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:55.989132 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:55.989401 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:55.990030 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:55.991942 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.018040 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1457/2943] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Sep 22 15:23:55 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:56.022197 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:56.033500 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f248b-32bb-4c00-944e-918c1004b92a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:56.088091 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:56.088688 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.088965 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.089234 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.089854 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:56.091785 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.112435 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1487/2944] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Sep 22 15:23:56 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:23:56.116416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:56.126505 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363a8e6-f86e-456b-ae0e-7fb898dc9208 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:56.131000 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:56.131616 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.131891 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.132159 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.132787 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:56.134364 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.145358 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1458/2945] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:56.149254 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:56.159715 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615e6f6-192e-4ac5-b4c7-81f0978d1123 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:56.164245 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:56.164883 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.165161 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.165431 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] PATH_INFO: `/v3/projects/fe26d63140c048f9a535ef2df67e9aae/users/315470e9863849229db89d745ca46977/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.181324 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe26d63140c048f9a535ef2df67e9aae, user_id=315470e9863849229db89d745ca46977, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:56.183991 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.196376 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1488/2946] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:56 2022] PUT /identity/v3/projects/fe26d63140c048f9a535ef2df67e9aae/users/315470e9863849229db89d745ca46977/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:56.200594 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:56.211362 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a175bf7-7cd6-4c09-b471-850b00ed3131 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:56.231297 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:56.269324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6fcd9-8a50-4b81-a269-e6923278fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:56.269859 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6fcd9-8a50-4b81-a269-e6923278fa49 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.270136 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6fcd9-8a50-4b81-a269-e6923278fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.270416 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6fcd9-8a50-4b81-a269-e6923278fa49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.271048 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6fcd9-8a50-4b81-a269-e6923278fa49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:56.272672 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6fcd9-8a50-4b81-a269-e6923278fa49 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.283131 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1459/2947] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Sep 22 15:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:23:56.287423 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:56.292790 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:56.299154 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e254328-ce68-4ffb-801d-0008b82c976b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:56.304141 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:56.304791 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.305066 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.305337 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] PATH_INFO: `/v3/projects/fe26d63140c048f9a535ef2df67e9aae/users/315470e9863849229db89d745ca46977/roles/5519f8e533ba4c5b8d81cdb709b861be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.307251 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe26d63140c048f9a535ef2df67e9aae, user_id=315470e9863849229db89d745ca46977, role_id=5519f8e533ba4c5b8d81cdb709b861be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:56.309863 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.320542 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1489/2948] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Sep 22 15:23:56 2022] PUT /identity/v3/projects/fe26d63140c048f9a535ef2df67e9aae/users/315470e9863849229db89d745ca46977/roles/5519f8e533ba4c5b8d81cdb709b861be => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:56.331035 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.331552 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.331697 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.359778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] MFA Rules not processed for user `315470e9863849229db89d745ca46977`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:56.360978 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:56.427281 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1460/2949] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:23:56.437988 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:56.440184 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:56.496266 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d305a-5550-467b-bc0e-c24392ee2c08 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:56.499865 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ea299-caad-4ec3-81dd-352f3f267dbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4e09c957a104f2299ab55d8283124c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:56.500463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ea299-caad-4ec3-81dd-352f3f267dbc service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:56.501231 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ea299-caad-4ec3-81dd-352f3f267dbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:56.501501 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ea299-caad-4ec3-81dd-352f3f267dbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:56.510182 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ea299-caad-4ec3-81dd-352f3f267dbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:56.511648 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ea299-caad-4ec3-81dd-352f3f267dbc service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:56.521814 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1490/2950] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:57.056366 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:57.116955 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc32595-0c44-468e-bc55-3535c964d49a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:57.121471 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:57.122090 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:57.122369 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:57.122643 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] PATH_INFO: `/v3/users/f3a60df3212b4e4b869aad12eda4bb66` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:57.123778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a60df3212b4e4b869aad12eda4bb66)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:57.126169 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:57.171895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a60df3212b4e4b869aad12eda4bb66'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.173521 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:57.179778 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a60df3212b4e4b869aad12eda4bb66'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.180490 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a60df3212b4e4b869aad12eda4bb66'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.195139 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a60df3212b4e4b869aad12eda4bb66'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.202609 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a60df3212b4e4b869aad12eda4bb66'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.210551 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1461/2951] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:57 2022] DELETE /identity/v3/users/f3a60df3212b4e4b869aad12eda4bb66 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:57.215138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:57.215460 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:57.215725 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:57.234233 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] MFA Rules not processed for user `f95719d40ecc461fa97c90ba29b8ba24`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:23:57.235463 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:57.293798 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1491/2952] 10.222.0.66 () {58 vars in 958 bytes} [Thu Sep 22 15:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:23:57.303399 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:57.304794 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:57.363677 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d3af0-a472-4a6a-ab53-525c66e1acc2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:57.367339 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60c809b5a47a4983be26116553934387', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:57.367936 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:57.368244 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:57.368540 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:57.377865 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:57.379324 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:57.388518 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1462/2953] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:23:57.654229 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:57.655872 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:57.668869 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e61f6-34d9-42a3-a42c-9a95b3bc1545 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:57.673967 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:57.674591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:57.674864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:57.675138 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] PATH_INFO: `/v3/projects/4c25e2747f134a32979baf911beef64f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:57.676316 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c25e2747f134a32979baf911beef64f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:57.678211 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:57.724699 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c25e2747f134a32979baf911beef64f', 'request_id': 'req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.731188 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c25e2747f134a32979baf911beef64f', 'request_id': 'req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.737291 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c25e2747f134a32979baf911beef64f', 'request_id': 'req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.748031 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] The token cache is being invalidate because project 4c25e2747f134a32979baf911beef64f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:57.748319 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.748949 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.750410 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1492/2954] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:57 2022] DELETE /identity/v3/projects/4c25e2747f134a32979baf911beef64f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:57.754998 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:57.764485 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f343df-f448-43c6-836c-21067299b351 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:57.819601 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:57.820228 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:57.820508 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:57.820781 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] PATH_INFO: `/v3/users/315470e9863849229db89d745ca46977` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:57.821895 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=315470e9863849229db89d745ca46977)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:57.823831 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:57.869282 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '315470e9863849229db89d745ca46977'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.876396 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '315470e9863849229db89d745ca46977'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.877009 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '315470e9863849229db89d745ca46977'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.888376 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:57.892473 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '315470e9863849229db89d745ca46977'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.901365 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '315470e9863849229db89d745ca46977'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:57.907851 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1463/2955] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Sep 22 15:23:57 2022] DELETE /identity/v3/users/315470e9863849229db89d745ca46977 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:23:58.169011 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:23:58.170469 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:23:58.183224 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552dbfee-d6b0-4ac1-9cfd-f96b7b409a64 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:23:58.196737 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:23:58.244521 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f95719d40ecc461fa97c90ba29b8ba24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0562937a351d4a8397127ac952e02625', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:23:58.245189 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:23:58.245479 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:23:58.245753 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] PATH_INFO: `/v3/projects/fe26d63140c048f9a535ef2df67e9aae` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:23:58.246931 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe26d63140c048f9a535ef2df67e9aae)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:23:58.248897 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:23:58.297603 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe26d63140c048f9a535ef2df67e9aae', 'request_id': 'req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:58.306911 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe26d63140c048f9a535ef2df67e9aae', 'request_id': 'req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:58.312882 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe26d63140c048f9a535ef2df67e9aae', 'request_id': 'req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:58.324013 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] The token cache is being invalidate because project fe26d63140c048f9a535ef2df67e9aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:23:58.324303 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:58.325069 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da35afa3-6f34-4b01-a56d-a29d6df0b6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:23:58.326591 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1493/2956] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Sep 22 15:23:58 2022] DELETE /identity/v3/projects/fe26d63140c048f9a535ef2df67e9aae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:24:07.903845 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:24:08.320864 dv-cinder-856463-5-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:27:40.001756 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:27:40.003232 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:27:40.141867 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e9691-ca7c-4d48-a140-01db3d6309d2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:27:40.198868 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f43e7a1e9840299697271428a67872', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83c0de024d3b4120a69df2088bded628', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:27:40.199496 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:27:40.199777 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:27:40.200047 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:27:40.201803 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:27:40.212909 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:27:40.214413 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571f091-52b3-41fc-8470-ff262b875de6 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:27:40.224765 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1464/2957] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Sep 22 15:27:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:27:50.220456 dv-cinder-856463-5-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}